Priority lanes and broadcasts

A marketing blast and a login code should never wait in the same line. Priority lanes give critical messages their own queues and worker pools. A broadcast, such as a campaign to 50 million users, needs a different approach. The product service must not make 50 million API calls. A separate expander service writes the work in chunks, and it slows down when the queues behind it are full.

The big idea

Shared infrastructure is fine. Shared saturation is not. Security and marketing can run on the same platform, as long as marketing can't fill the queues or workers that security depends on.

Think of an airport with a priority lane for connecting passengers. Everyone goes through the same security checkpoint, but the priority lane has its own staff. When the regular lanes back up, the priority lane still moves.

Walking through it

A flash sale sends a push to 50 million users. The steps:

1. The product service writes one broadcast job with an audience selector, such as "followers of creator X" or a segment query.
2. The expander reads the audience in pages, using keyset pagination, which means it asks for the rows after the last key it saw.
3. It applies preferences in bulk, then writes the survivors to the push queue in chunks of 1,000 to 10,000.
4. It watches the downstream queue depth, and it pauses when the queue is full. That is backpressure.

50,000,000 users in chunks of 10,000 is 5,000 chunks. The expander never holds the whole audience in memory, and it never blocks the product service.

The kill switch protects critical paths. Suppose the rule is: if the oldest message in the security queue is older than 2 seconds, pause the marketing consumers. Resume them when the security lane is healthy. Put the switch behind a config flag, so on-call can trip it during an incident without a deploy.

campaign job: audience = followers_of:creator_x
expander:     page 1 (keyset)  -> bulk preferences -> 10,000 items -> push queue
              page 2           -> ...
              waits when the push queue is full (backpressure)
security lane: oldest message 0.4 s   -> healthy, marketing keeps running
security lane: oldest message 2.5 s   -> kill switch: pause marketing consumers

For push, the provider's topic or audience APIs can send to a segment in one call, such as FCM topics. That is cheaper, but it gives you weaker personalization.

Each product also gets its own budget. Per-namespace token buckets, in Redis, stop a buggy games team from exhausting the SMS pool that auth codes use. Security templates are allowed by a list, so a product can't mark its marketing as security.

Common mistakes

• One queue for security and marketing. A campaign backlog then delays login codes.
• A product service enqueuing millions of calls. Use a broadcast job and an expander.
• No guard on the audience size. A runaway expander can flood the queues. Set a maximum audience and a kill switch.
• Building a complex fair scheduler first. A pause switch is easier to explain and to trip during an incident.

Check yourself

1. Why split 50 million users into chunks of about 10,000?

   The expander can apply backpressure, writing the next chunk only when the queues have room. It also never has to hold the whole audience at once.

2. When should the kill switch trip?

   When the security lane's oldest message is older than the set limit, such as 2 seconds. Marketing consumers pause until the security lane recovers.

Key takeaways

• Give critical messages their own queues and workers.
• Expand broadcasts in chunks, with backpressure from downstream queues.
• A kill switch behind a config flag protects the critical lane in an incident.
• Budget each product separately, so one team can't drain shared quotas.

Further learning

Next: templates and localization, which covers how each message is rendered before it is sent.