Chat connection layer

A chat client keeps one open connection to the service, so the server can push a message the moment it arrives. The connection layer holds those connections. WebSocket gateways terminate them. A session registry records which gateway holds each user's connection. Message storage sits behind this layer, and it doesn't need to know who is connected.

The big idea

Think of a switchboard operator. The operator knows which line each person is on, and they connect calls as they come in. The operator doesn't keep the conversations.

Gateways follow that rule. They hold connections, but no message history. If a phone reconnects to a different gateway, nothing is lost. The new gateway looks the user up in the registry, and history comes from storage.

The analogy breaks in one place. A switchboard holds almost no state. The registry does hold state, so its entries need expiry, or it will list users who are long gone.

Walking through it

The registry maps userId to devices, gateways and the time each was last seen. When a message must reach user B:

1. Look up B's active sessions.
2. Send the message to those gateways, through a pub/sub channel or a direct RPC.
3. If B has no sessions, queue it for sync on connect, and send a push notification through APNs or FCM.

Route through a lightweight pub/sub, partitioned by userId or gatewayId, so you don't broadcast to the whole fleet.

The protocol for a session looks like this:

CLIENT  AUTH { token }
CLIENT  MSG_SEND { channelId, clientMsgId, body }
SERVER  MSG_ACK { clientMsgId, messageId, ts }
SERVER  MSG_RECV { messageId, channelId, senderId, body, ts }
CLIENT  TYPING { channelId }

Capacity. Each idle WebSocket costs roughly 10 to 20 KB of memory.

Per gateway box: 100k connections × 15 KB ≈ 1.5 GB of connection state
Fleet-wide:      30M connections × 15 KB ≈ 450 GB
→ hundreds of gateway machines, not dozens

Sticky sessions only keep a user on one gateway for one TCP connection. The registry has to survive reconnects to a different machine.

Common mistakes

• Storing history on gateways. A reconnect lands on a different gateway and the history is gone.
• Assuming sticky sessions solve reconnects. They last only as long as one connection. Use the registry.
• Counting only messages per second. Connections set the gateway fleet size, and they can dominate the cost.

Check yourself

1. Why can any gateway serve a user after a reconnect? Gateways hold no history. The registry says where the user is now, and storage supplies the messages.
2. Why do open connections drive the cost? Each one holds memory and file descriptors on a gateway, so the fleet grows with the number of connections.

Key takeaways

• Split the connection layer, which is gateways plus a registry, from message storage.
• Keep gateways stateless about history, so any gateway can serve a reconnecting user.
• Give registry entries an expiry, so departed devices drop out.
• Route through partitioned pub/sub, not through a fleet-wide broadcast.

Further learning

The real-time delivery page covers the wider pattern of pushing data to clients.

Next, read about chat message ordering, which explains how each device keeps messages in order.