Retry policy and backoff with jitter

A retry is only safe when the operation is idempotent. A retry policy decides which errors are worth retrying, how long to wait between attempts, and when to give up. Without one, retries make bad outages worse.

Which errors to retry

Why jitter matters

Retrying immediately is the worst option. If a database blips for two seconds and 10,000 clients retry at once, the recovering database gets a synchronized wave the moment it returns, and falls over again.

Exponential backoff spreads attempts out. Jitter spreads the clients out too. Marc Brooker's analysis on the AWS Architecture Blog compared jitter strategies by simulation. It found that "full jitter", a random wait between zero and the exponential bound, finished the work with far fewer total calls than backoff without jitter.

base = 100 ms, cap = 10 s, full jitter: sleep = random(0, min(cap, base × 2^attempt))

attempt 0: bound 100 ms → sleep somewhere in 0 to 100 ms
attempt 1: bound 200 ms → 0 to 200 ms
attempt 2: bound 400 ms → 0 to 400 ms
attempt 3: bound 800 ms → 0 to 800 ms
attempt 4: bound 1.6 s  → 0 to 1.6 s
attempt 7: bound 12.8 s → capped at 10 s, so 0 to 10 s

The sketch below applies full jitter and respects the caller's deadline.

func withRetry(ctx context.Context, attempts int, fn func(context.Context) error) error {
	const base, maxDelay = 100 * time.Millisecond, 10 * time.Second
	var err error
	for attempt := 0; attempt < attempts; attempt++ {
		if err = fn(ctx); err == nil || !isRetryable(err) {
			return err
		}
		bound := min(maxDelay, base<<attempt)
		sleep := time.Duration(rand.Int63n(int64(bound))) // full jitter
		select {
		case <-time.After(sleep):
		case <-ctx.Done(): // don't retry past the caller's deadline
			return ctx.Err()
		}
	}
	return err
}

The deadline check matters. Retrying after the original caller has given up is pure waste.

Common mistakes

• Retrying instantly. It synchronizes clients into waves that hit the recovering system together.
• Retrying errors that can't change. A 400 or 404 fails the same way every time.
• Retrying a non-idempotent call. Backoff doesn't make a second charge safe.
• Ignoring the deadline. A retry that finishes after the caller left is wasted work.

Check yourself

1. Why wait the full Retry-After on a 429 instead of retrying sooner? The server has told you when it can accept the request. Retrying earlier only adds load it has already refused.

2. What does full jitter buy you that plain backoff doesn't? It spreads retries randomly across the window, so clients don't arrive at the server in synchronized waves.

Key takeaways

• Retry only errors that might change on the next attempt, and only idempotent operations.
• Exponential backoff with full jitter spreads retries out in time.
• Stop at the caller's deadline.
• Limit how much retrying the whole system does, which the next page covers.

Further learning

The AWS Architecture Blog post explains the jitter comparison and its simulation results.

Exponential backoff and jitter (AWS Architecture Blog)Update (May 2023): After 8 years, this solution continues to serve as a pillar for how Amazon builds remote client libraries for resilient systems. Most AWS SDKs now support exponential backoff and jitter as part of t…Amazon Web Services