Fan-out with pub/sub

Fan-out means one message reaching many recipients. In a chat system, a message to a channel with 2,000 online members must reach every gateway that holds one of those members' connections. The question is how the publisher finds those gateways without doing work per member.

The big idea

There are two designs. In route to user, the publisher looks up each recipient in the registry and sends to each one's gateway. In subscribe by interest, each gateway subscribes to the channels its users care about. The publisher sends once, and the backbone copies the message to each subscribed gateway. Each gateway then delivers to its local sockets.

Subscribe by interest is usually the better choice for group chat. The publisher writes once. The number of copies scales with the number of gateways, not the number of members.

Walking through it

Take a channel with 2,000 online members spread across 3 gateways.

Route to user:

for each of 2,000 members:
  look up gateway in the registry      → 2,000 lookups
  publish to that gateway's topic      → up to 2,000 publishes

Subscribe by interest:

publish once to channel:general        → 1 publish
backbone copies it to 3 subscribed gateways → 3 copies
each gateway writes to its local sockets    → 2,000 socket writes, spread across 3 machines

The second design moves the per-member work to the machines that already hold the sockets. Slack's architecture works roughly this way, with channel servers and gateway servers that subscribe to the channels they need.

Choosing a backbone matters:

Do the fan-out math before choosing:

A 100,000-member channel that receives 10 messages per second, delivered per viewer:
  100,000 × 10 = 1,000,000 socket writes per second

That is why very large channels, such as a live sports feed, are often batched. One frame carries 20 messages every 200 ms, or the feed is sampled or rate-limited per viewer.

Common mistakes

• Routing to each user for large groups. Lookups scale with members, not with gateways.
• Using plain Redis pub/sub for messages that must arrive. A subscriber that is down misses them. Use a persisted backbone.
• Forgetting the write multiplier. Large channels multiply every message by the number of members.

Check yourself

1. A 100,000-member channel gets 10 messages per second, and each viewer gets each message. How many socket writes per second?

   Answer: 1,000,000 (100,000 × 10).

2. Why does subscribe-by-interest beat route-to-user for a large group?

   Answer: the publisher writes once per message, and the backbone copies it once per gateway, not once per member.

Key takeaways

• Subscribe by interest lets the publisher write once, while gateways deliver to their own sockets.
• Choose a backbone by its delivery guarantees, not only its speed.
• The socket-write count grows with members times messages, so batch or sample very large channels.

Further learning

Next, see how heartbeats and resumes keep these connections honest, in connection lifecycle.