Cascading failures

A cascading failure starts with one part of the system getting slow, not broken. Callers wait, hold their threads and queue more work. Their own capacity drops, and the problem moves up the call graph. The cascade is dangerous because it spreads through the healthy parts of the system, which were not the cause.

The big idea

Think of a shop with one slow cashier. Customers queue behind that cashier, and the queue blocks everyone, including people buying one item who never needed that cashier. The analogy breaks in one place. A slow server does not send its customers away. Its threads stay blocked, and it cannot take new work at all.

Walking through it

Take a service with 200 worker threads. Each request calls a recommendations service that normally answers in 100 ms. By Little's law, the maximum throughput is the number of threads divided by the time each request holds one:

Healthy:   200 threads ÷ 0.1 s per request = 2,000 requests/s capacity

Degraded:  recommendations now takes 5 s
           200 threads ÷ 5 s per request   = 40 requests/s capacity

Nothing crashed. The recommendations service is only slow, yet your capacity fell by 98%. At a normal load of 1,500 requests per second, all 200 threads block within a fraction of a second.

Now every request fails, including the ones that never needed recommendations, such as checkout and login. Those requests cannot get a thread. Your callers see your service as slow, their threads fill up, and the failure climbs.

Two things make this worse. Retries add load to a dependency that is already struggling. If three layers each make four attempts, one user request can produce 64 calls at the bottom. Also, overloaded servers get less efficient. More garbage collection, lock contention and cache misses mean that useful work falls as load rises.

The patterns in this topic each cut one link in that loop.

Common mistakes

• Assuming slow means harmless. Slowness consumes threads and connections just as errors do.
• Looking only at error rates. Watch latency, and the saturation of threads and connections.
• Blaming the dependency alone. Retries and unbounded queues turn a slowdown into an outage.

Check yourself

1. A service has 200 threads. A call that took 0.1 s now takes 5 s. What is its capacity?

   Answer: 40 requests per second (200 ÷ 5), down from 2,000 (200 ÷ 0.1).

2. Why do checkout and login fail when only recommendations is slow?

   Answer: all 200 threads are blocked waiting on recommendations, so requests that never needed it cannot get a thread.

Key takeaways

• A cascade starts with slowness, and spreads through shared threads, connections and queues.
• Capacity equals concurrency divided by latency, so a slow call cuts capacity sharply.
• Retries multiply load on the failing part, and overload makes each request less efficient.
• Each later pattern breaks one link of this loop.

Further learning

Google's SRE book explains the same spiral, in which an overloaded server fails its health check and the cluster falls over one machine at a time:

Next, see how timeouts and deadlines stop a slow call from holding resources forever in timeouts and deadlines.