Latency SLOs and histograms

A latency SLO says what share of requests must finish within a threshold. For example, 99% of requests finish in under 300 ms. It works like an availability SLO: good events divided by total events. Here, a good event is a request that was fast enough.

Good and total

• Good: requests whose duration is at or below the threshold.
• Total: all eligible requests.
• Optional: exclude 5xx responses from the latency measure, so one failure does not count twice. Write down which choice you made.

Why buckets matter

A histogram counts requests into buckets, such as "at most 100 ms" and "at most 300 ms." You can read "under 300 ms" exactly only if 300 ms is a bucket edge. If it is not, the number you read is for some other threshold.

Here are made-up counts to show the arithmetic:

Bucket le=0.1    ->    620 requests
Bucket le=0.3    ->  9,880 requests
Bucket le=+Inf   -> 10,000 requests

Good ratio at 300 ms = 9,880 / 10,000 = 98.8%

This query computes the same ratio in Prometheus, using a classic histogram. It is the fraction of requests under 300 ms:

sum(rate(http_request_duration_seconds_bucket{le="0.3"}[5m]))
/
sum(rate(http_request_duration_seconds_count[5m]))

Pick bucket boundaries so your SLO threshold sits on one of them. Native histograms, where your metrics system supports them, give finer resolution.

Do not average percentiles

Never build a latency SLO on the average of each pod's p99. The p99 of several pods' p99 values is not the p99 of all requests. Compute the ratio from bucket counts instead, which combine correctly across pods. The metrics and alerting page covers percentiles in more depth.

Common mistakes

• Putting the threshold between buckets. You measure a different threshold than the one you promised.
• Averaging p99s across pods. The result does not describe the users.
• Counting 5xx as both an availability failure and a latency failure. Pick one rule and document it.

Check yourself

1. Using the buckets above, what share of requests finished under 300 ms? Answer: 9,880 of 10,000, or 98.8%.
2. Why must 300 ms be a bucket edge? Answer: a histogram only counts requests per bucket, so it can only answer exactly at an edge.

Key takeaways

• A latency SLO is a good-over-total ratio with a time threshold.
• Put the threshold on a histogram bucket edge.
• Do not average percentiles across pods.
• Next, rolling out SLOs explains how to introduce all of this on a team that has none.

Further learning

Histograms and summaries (Prometheus docs)Prometheus project documentation for Histograms and summariesprometheus.io

Gil Tene's talk shows why averages and naive percentiles hide the slow requests users actually feel, including coordinated omission.