Insufficient Resources
错误信息
0/3 节点是 available: 3 Insufficient cpu
快速摘要
cluster 做不 有 enough available CPU or memory 到 schedule Pod's requested resources.
为什么发生
Pod 请求更多 CPU or memory than 任何节点有 available
cluster 是 at capacity — 所有节点是 fully allocated
Resource 请求是 设置也 高 for 实际需要
节点是 cordoned or drained, reducing available capacity
最小示例
resources:
requests:
cpu: 4 # no single node has 4 CPU free
memory: 16Giresources:
requests:
cpu: 1 # reasonable request
memory: 2GiPod 请求 4 CPUs 但否 节点有 那个 much free. fix reduces 请求到 realistic 值.
如何诊断
检查节点 resources:
kubectl 描述节点and look at Allocatable 对比 AllocatedView Pod resource 请求:
kubectl 获取 pods -o yamland 检查 resources.请求检查 for cordoned 节点:
kubectl 获取节点Use
kubectl 顶部节点到 see current resource usage
如何修复
Reduce Pod's resource 请求到 match 实际需要
添加更多节点到 cluster (scale cluster)
Uncordon 节点那个是 accidentally cordoned
删除 or migrate Pods 那个是 over-consuming resources
如何预防
设置 realistic resource 请求 based on 实际 usage
Use cluster autoscaler 到添加节点 automatically 当需要
Monitor cluster capacity and alert 之前它 是完整