Traffic and messaging
Real-time delivery
How a server pushes updates to clients that did not ask, from polling to WebSockets, and the gateway, pub/sub and reconnect design that serves millions of open connections. Start here for the real-time map.
HTTP was built for the client to ask and the server to answer. Many products need the reverse. A chat message should appear the moment it is sent. A ride app should show the car moving. An LLM should stream tokens as it writes them. In each case the server has news, and it has to push that news to a client that did not ask for it.
This topic covers the transport options, from simple polling to WebSockets. It then covers the architecture that delivers messages to millions of open connections: gateways, a connection registry, a pub/sub backbone, heartbeats and resumable reconnects.
Choosing a transport
| Short polling | Long polling | SSE | WebSockets | WebTransport | |
|---|---|---|---|---|---|
| Direction | Client pulls | Client pulls, server holds | Server to client | Both | Both |
| Delivery delay | Up to the poll interval | Near-instant | Near-instant | Near-instant | Near-instant |
| Overhead per message | Full HTTP request | Full HTTP request | A few bytes | 2 to 14 bytes | Small |
| Reconnect and resume | Not needed | Manual | Built in | Manual | Manual |
| Best for | Refresh every 30 seconds or more | Legacy fallback | Feeds, notifications, token streams | Chat, games, collaboration | Games, media, lossy networks |
Pick the simplest option that works. Polling is fine when updates are slow. Server-Sent Events fit one-way streams. WebSockets fit two-way traffic with frequent small messages.
Concepts in this topic
Read them in order. Each one builds on the last.
| Step | Concept | What it answers |
|---|---|---|
| 1 | Polling and long polling | How do you get updates from a client that only pulls? |
| 2 | SSE and WebSockets | How does the server push over a long-lived connection? |
| 3 | Gateways and connection registry | Which machine holds user 42's connection? |
| 4 | Fan-out with pub/sub | How does one message reach thousands of connected users? |
| 5 | Connection lifecycle | How do heartbeats, resumes and reconnects avoid losing messages? |
| 6 | Presence and typing | How do you show who is online without drowning the system? |
Common mistakes
- Using sockets by default. A feed that refreshes every minute does not need a persistent connection.
- Keeping sockets open in the background on mobile. Phones suspend background apps, and an always-on radio drains the battery. Use platform push for background updates, and treat it as a signal to sync.
- Shipping gateway code that changes daily. Every gateway deploy disconnects its users. Keep business logic in stateless services.
- Forgetting reconnects. Clients disconnect constantly. Plan for a resume and a jittered retry from day one.
Key takeaways
- Short polling is simple but wasteful. Long polling, SSE and WebSockets push updates as they happen.
- Real-time systems split into thin, stateful connection gateways and stateless application services, linked by pub/sub.
- Heartbeats detect dead connections, and sequence numbers make reconnects lossless.
- Presence and typing are ephemeral. Use keys with a TTL, and fan out only to people who can see them.
- Plan for reconnect storms and deploys, not just steady-state load.
Further learning
Slack's engineering team describes how gateway servers, channel servers and presence servers deliver messages to millions of clients:
Real-time Messaging (Slack Engineering)Did you know that ground stations transmit signals to satellites 22,236 miles above the equator in geostationary orbits, and that those signals are then beamed down to the entire North American subcontinent? Satellite…slack.engineering
Discord's account of scaling Elixir to 5 million concurrent users covers the same gateway design in production:
How Discord scaled Elixir to 5,000,000 concurrent usersHow Discord Scaled Elixir to 5,000,000 Concurrent Usersdiscord.com
Start with polling and long polling to see the simplest option first.
Related
Case study: chat system
Design a WhatsApp or Slack style messaging system. Connection gateways, per-conversation ordering, wide-column storage and large-channel fan-out are the parts that matter.
Case study: notification system
Designing push, email and SMS delivery at scale, covering the pipeline, preference checks, idempotency, provider rate limits, retries and priority lanes.
Polling and long polling
How a client gets updates by asking, why short polling wastes almost every request, and how long polling holds each request open until there is news.
SSE and WebSockets
Server-Sent Events push a one-way stream over plain HTTP with built-in resume, WebSockets give full-duplex frames, and WebTransport adds streams and datagrams over HTTP/3. Choose by direction and message rate.
Gateways and connection registry
Why real-time servers split into thin stateful gateways and stateless services, how a TTL registry finds the gateway holding a user, and how many connections one machine can hold.