Circuit breakers

A circuit breaker wraps a call to a dependency and tracks recent outcomes. When failures pass a threshold, it stops sending calls and fails them at once. A timeout limits how long one call can hurt you. A circuit breaker stops many calls from hurting you at the same time.

The big idea

A household circuit breaker cuts power when the wiring is overloaded, so the wiring is protected. A software breaker does the same for a dependency. Failing calls return immediately, your threads stay free, and the dependency gets time to recover.

The analogy breaks in one place. A household breaker stays off until someone resets it. A software breaker tests the dependency again on its own, in the half-open state described below.

Walking through it

A breaker moves between three states:

• Closed. Calls go through. The breaker counts failures over a rolling window.
• Open. The failure rate passed the threshold. Calls fail immediately, without touching the network.
• Half-open. After a cool-down, a few trial calls go through. If they succeed, the breaker closes. If they fail, it opens again.

The numbers in the diagram are the defaults that Netflix's Hystrix library popularized around 2012. Hystrix is now in maintenance mode. Resilience4j is a common successor on the JVM. Service meshes such as Envoy and Istio apply the same idea per host, through outlier detection, which ejects one bad host from the load-balancing pool.

A minimal breaker, with a demo that calls a dependency that always fails:

class CircuitBreaker {
  private failures = 0
  private openedAt: number | null = null

  constructor(private threshold = 3, private coolDownMs = 5_000) {}

  async call<T>(fn: () => Promise<T>): Promise<T> {
    if (this.openedAt !== null) {
      if (Date.now() - this.openedAt < this.coolDownMs) throw new Error("circuit open")
      this.openedAt = null // cool-down over: let a trial call through
    }
    try {
      const result = await fn()
      this.failures = 0
      return result
    } catch (err) {
      if (++this.failures >= this.threshold) this.openedAt = Date.now()
      throw err
    }
  }
}

const breaker = new CircuitBreaker(3, 5_000)
const flaky = async () => { throw new Error("timeout") }
for (let i = 1; i <= 4; i++) {
  try {
    await breaker.call(flaky)
  } catch (e) {
    console.log(`call ${i}: ${(e as Error).message}`)
  }
}

Output:

call 1: timeout
call 2: timeout
call 3: timeout
call 4: circuit open

The third failure trips the breaker. The fourth call never reaches the dependency. This is a simplified sketch. Production breakers use error rates over sliding windows, cap the trial calls in half-open, and count timeouts as failures.

Common mistakes

• Counting consecutive failures only. A single success can hide a 60% error rate. Use rates over a window.
• Ignoring timeouts. A slow call that times out is a failure and should count as one.
• One breaker for a dependency with one bad host. Pair a per-dependency breaker with per-host outlier detection.
• No plan for the open state. Decide what callers get back: an error, a cached value or a degraded page.

Check yourself

1. Why does an open breaker help a dependency that is already failing?

   Answer: it stops new calls, so the dependency gets room to recover, and your threads are not held by calls that will time out.

2. What happens in the half-open state if the trial call fails?

   Answer: the breaker opens again, and the cool-down starts over.

Key takeaways

• A circuit breaker fails calls fast when a dependency's failure rate is high.
• It has three states: closed, open and half-open.
• Use error rates over a window, and count timeouts as failures.
• Pair breakers with a plan for what callers get when the circuit is open.

Further learning

A breaker limits calls to one dependency. A bulkhead limits how much capacity any dependency can take. Read bulkheads and isolation next.