Worked example - a 5x launch

This example plans a checkout API for a launch. Today's peak is 2,000 requests per second. The launch target is 10,000, which is 5 times the peak. The p99 target is 300 ms. Each request costs about 25 ms of application CPU, and a cache miss adds a 15 ms database query. The steps below use those numbers.

Step 1: application compute

At 2,000 requests per second, with about 25 ms of work per request, the work needs 2,000 × 0.025 = 50 busy cores. Real efficiency is worse, because of garbage collection and stalls. So measure it.

Today the service runs 20 pods with 2 cores each at about 60% CPU at peak. That is 20 × 2 × 0.6 = 24 effective cores. Scaling linearly to 5 times the load suggests about 24 × 5 = 120 cores, before headroom.

Step 2: headroom for a zone loss

The target is 75% utilization at launch peak, even after losing a zone. Two surviving zones must carry 10,000 requests per second at 75%.

10,000 / 0.75 = 13,333 requests/s of capacity on two zones
x 3/2 for three zones = about 20,000 requests/s of raw capacity

Provision for about 20,000 requests per second of raw happy-path capacity, or size each zone so that two of them carry the peak at 75%. Either framing works, as long as the zone failure is in the arithmetic.

Step 3: the database

Here the miss rate matters most. At 10,000 requests per second, a 40% miss rate sends 10,000 × 0.40 = 4,000 queries per second to the database.

Suppose the primary handles 6,000 queries per second before its knee in staging. Then the headroom is 6,000 / 4,000 = 1.5×. That is too tight for a launch.

The options are to raise the cache hit rate. A miss rate under 20% gives 10,000 × 0.20 = 2,000 queries per second. Other options are read replicas for read-heavy paths, pre-warming the cache, or scaling the database. Prove the choice with a load test that includes both cold and warm caches.

Step 4: prove it

Run open-model k6 steps through 2k, 5k, 8k, 10k and 12k requests per second, against a production-like environment. Gate on p99 below 300 ms, and an error rate below 0.1%. Soak at 10k for two hours. Then run a chaos test: kill one zone's worth of pods at 8k requests per second, and confirm recovery stays within the SLO.

Step 5: launch controls

Capacity is necessary, but it is not a defence against a 10 times surprise. Add admission control and load shedding, so the system degrades gracefully. Rate limiting belongs here too. See the rate limiting page.

Common mistakes

• Sizing from a spreadsheet alone. The model is not evidence until a test confirms it.
• Ignoring the miss rate. It decides how much load reaches the database.
• Scaling the app to fix a database limit. More pods only add more queries.

Check yourself

1. Why is a 40% miss rate at 10,000 requests per second a problem? Answer: It sends 4,000 queries per second to the database, which leaves only 1.5 times headroom against its knee.
2. What is the first control for an unexpected 10 times spike? Answer: Admission control or load shedding, because capacity alone cannot absorb a spike that large.

Key takeaways

• Work from measured cores per request, then scale and add headroom for a zone loss.
• The cache miss rate sets the load on the database. Check it first.
• Prove the numbers with an open-model load test, a soak and a zone-loss test.
• Keep admission control ready for spikes beyond the plan.

Further learning

The general planning method is on the capacity planning hub, which links every step in this example.