Timeouts and deadlines

Every network call needs a timeout. Without one, a call to a stuck dependency can hold a thread indefinitely. A timeout limits how long one call may wait. A deadline is an absolute cutoff for an entire request. A deadline applies to every hop, so a slow step near the end cannot push the whole request past the user's budget.

The big idea

Think of a relay race. Each runner has a lap time, which is a timeout. The team also has to finish by 7:30, which is a deadline. A runner who knows the team's remaining time can skip a lap that cannot finish in time.

Pass the remaining time to each hop, not a clock time. Remaining time works even when the machines' clocks disagree. gRPC does this automatically, so the callee knows how much time the caller has left.

Walking through it

Setting a timeout from data. A common start is the dependency's p99.9 latency plus some margin. If a call's p99.9 is 300 ms, a 500 ms timeout cuts off almost only the calls that are truly stuck. A stuck call then costs half a second, not a minute.

Why per-call timeouts are not enough. Suppose the user's request has a 1-second budget. It calls service A, which calls B, which calls C. Each hop has a 1-second timeout. If C hangs, B waits 1 second, A waits 1 second, and the work keeps going after the user has given up. A single deadline stops every hop once the budget is spent.

This sketch shows how a service might cap its own share of a deadline. It assumes a pricing client exists:

func (s *Server) GetProduct(ctx context.Context, id string) (*Product, error) {
	// ctx carries the caller's deadline. gRPC passes it across the network,
	// so this service knows how much time is left.
	ctx, cancel := context.WithTimeout(ctx, 300*time.Millisecond) // cap our own share
	defer cancel()

	price, err := s.pricing.GetPrice(ctx, id) // fails fast once the deadline passes
	if err != nil {
		return nil, err
	}
	return &Product{ID: id, Price: price}, nil
}

Deadlines also let a server skip work that nobody will read. If 5 ms remain and the query needs 50 ms, return an error immediately.

Common mistakes

• No timeout on an HTTP client. Some libraries wait forever by default, or a minute or more.
• Giving every hop the full budget. Each hop then waits longer than the user will.
• Doing expensive work after the deadline has passed. Check the remaining budget before you start.

Check yourself

1. Why is a 1-second timeout at every hop worse than one 1-second deadline?

   Answer: each hop keeps working until its own timer fires, so the chain can run far past the user's budget. A deadline stops every hop when the budget is spent.

2. Why pass the remaining time instead of an absolute clock time?

   Answer: remaining time works even when the machines' clocks disagree, so every hop does not need synchronized clocks.

Key takeaways

• A timeout limits one call. A deadline limits the whole request.
• Set timeouts from the dependency's measured latency, not a guess.
• Pass the remaining budget downstream, and check it before expensive work.
• Every outbound call needs a timeout.

Further learning

Retries are the next risk. Each retry adds load, so the next page covers when they help and when they make an outage worse, in retry budgets and hedging.