Cells and regional isolation

Most outages come from your own changes, not from nature. A bad deploy or a bad configuration push is the usual cause. If you push that change to every region at once, you have built an expensive way to fail everywhere at the same moment. Regional isolation and cells limit that damage.

The big idea

A cell is a complete, independent copy of the stack that serves a fixed slice of users. A thin routing layer maps each user to a cell. A failure inside one cell touches only that cell's users. A cell works like a watertight compartment in a ship: one breach floods one compartment, not the whole hull.

The analogy breaks because cells must be the right size. Too few cells means a large blast radius. Too many means more copies of the stack to run.

Walking through it

With 10 cells per region, a bad deploy reaches at most 10% of that region's users if it is caught in the first cell. Deployments go to one cell first, then a few, then the rest, with bake time in between:

Deploy v2.4 to cell 1 of 10       -> 10% of users in the region run v2.4
bake time, watch errors            -> errors spike, roll back cell 1
result                             -> 10% of one region affected, not 100%

If the bake time looks clean, the rollout continues to cells 2 and 3, then to the rest. The same pattern applies to regions: change one region, let it bake, then move to the next.

Regional isolation is what makes this possible. Regions should share as little as possible:

• Separate deployments and separate control planes.
• No synchronous cross-region calls in the request path. A region must not depend on another region to serve its own traffic. Otherwise one region's failure becomes both regions' failure.
• Changes roll out region by region, with bake time between them.

Slack described moving to a cellular architecture where each availability zone acts as a cell that can be drained within minutes. That mattered after incidents where one degraded zone affected everyone. The complementary idea is static stability: keep capacity already in place, so a failure doesn't force you to scale up under stress. The resilience page covers shuffle sharding, which limits the impact of individual large customers in the same way.

Common mistakes

• Pushing config to every region at once. One bad value then breaks everywhere. Stage it, and let each step bake.
• Cells that share a database. The cells then fail together. Each cell needs its own data path.
• Making a region depend on another region. A cross-region call in the request path couples both regions' availability.

Check yourself

1. A bad deploy reaches 1 of 10 cells in a region. What share of that region's users are affected?

   10%.

2. Why must a region avoid synchronous calls to another region in the request path?

   A failure in the other region then becomes a failure in this one, which breaks regional isolation.

Key takeaways

• Most outages come from your own changes, so limit the blast radius of changes.
• A cell is an independent stack for a fixed slice of users.
• Roll out to one cell or region first, let it bake, then continue.
• Regional isolation means no synchronous dependency on another region.

Further learning

Slack's migration to a cellular architecture describes the change in practice, including draining a zone within minutes.

Slack's migration to a cellular architectureSummary In recent years, cellular architectures have become increasingly popular for large online services as a way to increase redundancy and limit the blast radius of site failures. In pursuit of these goals, we hav…slack.engineering

The AWS cell-based architecture talk explains how cells cap blast radius, and how to route and size them.

Next: failover testing, which shows how to prove that the regional plan works.