Capacity planning and load testing

Capacity planning answers a blunt question. Will this system absorb next month's traffic, the launch spike, and a zone failing in the middle of that spike, without breaking its latency targets? Guessing from average CPU charts is how teams learn the answer during an outage. Measuring demand, modelling queues and proving limits with load tests tells you before users do.

This is the operational half of estimation. Back-of-envelope maths sizes the first deploy. Capacity planning keeps the fleet honest as growth, features and dependencies change. On Kubernetes it pairs with requests, limits and autoscaling.

The concept pages

Autoscaling is not a plan by itself

Horizontal pod autoscalers and cloud autoscaling groups react to lagging signals. By the time CPU averages cross a threshold, queues may already be deep. Capacity planning sets the floor, the minimum replicas that absorb the fast part of a spike. It also sets the ceiling, the maximum before you overload a shared database. Autoscaling fills the middle.

When the bottleneck is a primary database, more app pods make the outage worse. Name the limiting resource in one sentence. "We are connection-bound at 4k QPS" is a plan. "The API will scale" is not.

Revisit the numbers

Capacity is not an annual exercise. Revisit it when traffic grows by about 20%, when you add a heavy endpoint, when a dependency changes its limits, or when p99 drifts up at the same request rate. That drift is an early sign that you are moving toward the knee. Run a load test on the top three user journeys every quarter, and store the report with the service. The next on-call engineer should see the last proven limit.

Common mistakes

• Load-testing only the cache-hit path. The real traffic includes misses.
• Using closed-loop virtual users. They slow down with the system and report less delay than users feel.
• Provisioning for an average Monday. Peaks and launches decide the size.
• Forgetting deploy surge. A rolling update needs room for the new pods before the old ones leave.
• Ignoring a dependency's rate limit. It is often the real ceiling.

Key takeaways

• Forecast the peak and the events, not just the average.
• Plan for about 70 to 80% utilization on the scarce resource, not 95%.
• Little's law connects requests per second, latency and concurrency, and sizes pools.
• Test with realistic, open-model traffic, and find the knee.
• Autoscaling needs a floor, a ceiling and a named bottleneck. It does not replace planning.

Further learning

Google's SRE chapter on overload connects admission control, load shedding and graceful degradation to capacity limits.

Handling Overload (Google SRE book)Learn how load balancing, criticality handling, client side throttling protect systems from overload, ensuring fault-tolerant performance in distributed systems.sre.google

Next, start with the demand side of the forecast: demand forecasting.