Shifting traffic between regions

Moving users from one region to another is a dial, not a switch. Weighted DNS lets you send 90% of traffic to one region and 10% to the other, watch the results, and turn the dial back if something goes wrong. This page is that runbook with the numbers filled in.

The big idea

A region move is a controlled blast radius. You expose a small slice of users to the new region, check the results, and widen the slice only when the numbers hold.

The dial analogy breaks in one place. DNS moves requests, not data. If the new region's database lags behind, moving traffic makes the lag visible to users. The data problem is still yours to solve.

Preconditions

• Both regions serve the app, and the data strategy is decided.
• Each region's load balancer has a health check.
• The TTL is lowered to 60 seconds at least a day before the change, or earlier if you can.
• Dashboards show request rate, error rate, latency, database lag and saturation for each region.
• Someone owns each region, and the rollback weight is written down.

The steps

1. Confirm the new region is healthy with a tiny weight, such as 5%.
2. Set weighted records for the API hostname: 90 for the old region, 10 for the new. Alias each to its regional load balancer.
3. Watch for 10 to 15 minutes. Check error budgets, p99 latency, dependency errors and queue alarms.
4. Step to 70/30, then 50/50, then 0/100 as confidence grows. Pause on any burn-rate alert.
5. Leave the TTL short for a day after the shift, then raise it.

Walking through a real rollback

At 50/50, the new region shows a 2% error rate, while the old region sits at 0.1%.

DNS did its job here. It made the blast radius a dial you can turn back.

Common mistakes

• Moving traffic before lowering the TTL. Caches hold the old answer for the full TTL.
• Blaming DNS for data lag. Fix replication or the write path.
• Jumping from 50/50 to 0/100. Each step needs a watch period.
• No written rollback weight. Under pressure, nobody remembers which weight was safe.

Check yourself

1. Why lower the TTL a day before the shift? Caches that fetched the old record expire quickly, so the new weights take effect fast.
2. At 50/50 the new region's error rate climbs, and its replicas lag 12 seconds. What is the first move? Roll back to 90/10, then fix replication or send writes to the primary region.

Key takeaways

• A traffic shift is a dial with a written rollback point.
• Lower the TTL before the shift, and watch each step for 10 to 15 minutes.
• DNS moves requests, not data. Check replication before you move users.

Further learning

For the routing policies and TTL details behind this runbook, read DNS routing and TTL. For the multi-region design choices, see the multi-region page.