The Silent Killer of Latency
Many teams set CPU Limits thinking they're being responsible, then spend weeks debugging mysterious latency spikes in their APIs. The culprit is almost always CFS (Completely Fair Scheduler) Quota — the Linux kernel mechanism that enforces CPU Limits.
How CFS Quota Works
The Linux kernel divides time into fixed periods of 100ms (by default). For each period, a container is allocated a quota of CPU time proportional to its Limit.
- A Limit of
1000m(1 full core) = 100ms of CPU time per 100ms period. - A Limit of
200m(0.2 cores) = 20ms of CPU time per 100ms period.
Once a container exhausts its quota for the period, the kernel pauses all its threads until the next period begins — even if the Node's CPUs are sitting completely idle.
A Concrete Example
Suppose your web server has a CPU Limit of 200m and receives an HTTP request that requires 30ms of CPU work to process:
- The container starts processing. It uses 20ms of CPU — its full quota for this period.
- The kernel pauses all threads for the remaining 80ms of the period.
- The next period begins. The container resumes and finishes the remaining 10ms of work.
Actual CPU work: 30ms
Wall-clock time experienced by the user: 110ms
Latency penalty introduced by the Limit: 80ms
This is not a bug in your application. It is the kernel doing exactly what you asked. The Limit you set is causing the latency.
The Practical Recommendation
For latency-sensitive workloads (APIs, user-facing services), many experienced platform engineers recommend setting CPU Requests but omitting CPU Limits entirely. The Request still ensures the Scheduler places the Pod on a Node with sufficient capacity. Without a Limit, the container can burst to use spare CPU cycles on the Node without being artificially throttled.
The trade-off: without Limits, a runaway process could consume all spare CPU on a Node, impacting neighbors. Mitigate this by using LimitRange objects to set namespace-level defaults, and by ensuring your Nodes are not over-committed.
A common SRE interview question: "Your API's p99 latency is high but CPU utilization looks normal. What would you check?" A strong answer includes: check container_cpu_cfs_throttled_seconds_total in Prometheus. High throttling with normal-looking average utilization is the classic CFS quota signature — the container is being paused by the kernel even though it's not "busy" by average metrics. Follow up by explaining the 100ms period model.