Resource requests, limits and QoS

Every container should declare requests, which are what the scheduler reserves for it, and usually limits, which are the hard ceiling. The two numbers do different jobs. The scheduler places pods using requests, not limits. The kernel enforces limits, and the way it enforces them differs for CPU and memory.

The big idea

Think of a hotel. A request is the room you booked, and the hotel must hold it for you. A limit is the most you can use while you stay. A hotel with 100 rooms can refuse the 101st booking, even if half the guests are out. That is what the scheduler does with requests, whatever the pods actually use.

Walking through it

resources:
  requests:
    cpu: "250m"      # 0.25 of one core
    memory: "256Mi"
  limits:
    cpu: "1"
    memory: "512Mi"

A millicore (m) is one-thousandth of a CPU. 250m entitles the container to 250 ms of CPU time per second of wall clock, on one core. Memory is a hard byte count, where Mi means mebibytes. CPU is compressible, so the kernel can slow it down. Memory is not, so the kernel must kill something.

The enforcement is different:

• Over the CPU limit: the kernel throttles the container. The process keeps running but waits for the next scheduling period, usually 100 ms. Latency spikes, and nothing crashes. The metric is container_cpu_cfs_throttled_seconds_total.
• Over the memory limit: the cgroup OOM killer sends SIGKILL. The exit code is 137. There is no chance to flush buffers.

Here is what that looks like in practice. An API pod requests 100m CPU and has a limit of 100m. Under a spike, it needs about 0.4 cores for JSON serialization. The kernel throttles it to 0.1 core. p99 jumps from 40 ms to 800 ms, and the Deployment still looks healthy. The fixes are to raise the limit, remove the CPU limit and rely on the request, or add replicas. A separate worker with a 256 MiB memory limit that loads a 300 MiB file dies with 137. Raise the limit, or stream the file.

Pods also get a QoS class, which decides who is evicted first when a node is under pressure:

For latency-critical serving, Guaranteed with memory request equal to limit is a common choice. A Burstable pod with a small memory request looks optional to the eviction manager.

Right-sizing starts from measurements. Look at the p95 working set and CPU usage in Prometheus, then set memory near the p95 with 25 to 50% headroom, and CPU near the p95 usage so autoscaling has a stable baseline.

Namespaces add two guardrails. A LimitRange supplies defaults when a resources: block is missing. A ResourceQuota caps the sum of requests in the namespace. A deploy that fails with Forbidden: exceeded quota has hit the quota, not a mysterious scheduler.

Common mistakes

• Setting CPU limits far below the real burst. The app looks slow with low average CPU.
• Leaving requests off. The pod is BestEffort, which is evicted first. Autoscaling on CPU also stops working, because it measures usage against the request.
• Reading node memory as the limit. A pod is OOM-killed against its own cgroup limit, even when the node has memory free.

Check yourself

1. A pod requests 100m and is limited to 100m. It uses 0.4 cores under a spike. What happens? It is throttled to 0.1 core. The process stays up, but requests wait, and latency rises.
2. A pod exits with 137. What limit was hit? Its memory limit. The kernel's OOM killer sent SIGKILL.

Key takeaways

• Requests drive scheduling and autoscaling. Limits drive throttling and OOM kills.
• CPU over its limit is throttled. Memory over its limit is killed (137).
• Set requests from measured usage, and choose a QoS class on purpose.

Further learning

The node-side view, including the cgroup budget, is in container throttling and cgroups. The probes that decide when a pod is ready are in health probes.

Kubernetes in production