Load testing for the knee

A load test sends synthetic traffic at a system, so you can watch latency, errors and saturation as demand rises. The goal is not a vanity requests-per-second number. The goal is to find the knee: the load where p99 latency turns sharply upward, or where successful throughput stops growing.

Tools

Realistic traffic

Match the read-to-write mix, payload sizes, authentication and cache hit rate to production. Prefer open models, with a constant arrival rate, when you ask capacity questions. A closed model has each virtual user wait for the previous response. That slows the test down when the system slows, which hides the delay users would feel. This is called coordinated omission.

Ramp in steps, and hold each step long enough for caches, garbage collection, autoscalers and connection pools to settle. That often takes minutes, not seconds. Then run a soak test at planned peak for hours, to catch leaks and disk growth. Separately, find the breaking point, to see how failure looks.

import http from "k6/http";

export const options = {
  scenarios: {
    ramp: {
      executor: "ramping-arrival-rate",
      timeUnit: "1s",
      preAllocatedVUs: 200,
      maxVUs: 2000,
      stages: [
        { target: 500, duration: "5m" },
        { target: 1000, duration: "5m" },
        { target: 2000, duration: "5m" },
      ],
    },
  },
  thresholds: { http_req_failed: ["rate<0.01"], http_req_duration: ["p(99)<300"] },
};

export default function () {
  http.get("https://api.example.com/v1/catalog");
}

k6 prints a summary at the end, and each threshold is marked as passed or failed.

Walking through a knee hunt

Watch the load generator too. If dropped_iterations climbs, the generator hit its virtual-user ceiling. The test then under-delivered the load it claimed.

Reading the knee

Plot achieved successful requests per second, and p99 latency, against offered load. Below the knee, successful throughput tracks the offered load, and p99 stays flat. Above it, p99 rises and throughput stops tracking the load. Queues grow and timeouts appear. Correlate with the resource that saturated first. That is your bottleneck hypothesis.

Common mistakes

• Testing only the cache-hit path. Production also sends misses.
• Using a closed model and declaring victory. The test measures its own slowdown.
• Stopping at one request rate. One point shows nothing about the knee.

Check yourself

1. The generator's dropped_iterations count keeps climbing. What does it mean? Answer: The generator hit its virtual-user ceiling, so the real offered load was below the target.
2. Why use an open model for capacity questions? Answer: A closed loop slows down with the system, so it hides the delay that real users would feel.

Key takeaways

• Find the knee, not a single requests-per-second number.
• Use open arrival rates, realistic traffic and step ramps.
• Soak at peak for hours, and find the breaking point once.
• Read the generator's metrics, and correlate with the first resource that saturates.

Further learning

A clear walkthrough of which load-test metrics matter when hunting bottlenecks.

The k6 documentation covers arrival-rate executors, thresholds and CI integration.

k6 documentationGrafana k6 | Grafana k6 documentationGrafana Labs

Next, plan for what happens when a zone fails during the peak: headroom for failure.