Chat requirements and scale

A chat system delivers messages between people in near real time, stores their history, and waits for recipients who are offline. WhatsApp, Slack, Discord and iMessage differ in product, but their core design is the same. Before you draw any boxes, state the requirements and the rough size. Those two things decide almost every later choice.

The big idea

Two chat products share a skeleton and diverge hard in two places. One is fan-out, how a message reaches many people. The other is search and encryption, whether the server can read message bodies.

So ask early whether the product is WhatsApp-style, with end-to-end encryption and small groups, or Slack- or Discord-style, with searchable history and large rooms. That single answer changes storage features and fan-out.

Walking through it

Functional requirements:

• One-to-one and group chat. Start with groups of up to a few hundred, and treat large channels separately.
• Send a message, deliver it to online members, and queue it for offline ones.
• Delivery and read receipts, showing sent, delivered and read.
• Multi-device sync, so the same user on a phone and a desktop sees the same history.
• History with pagination, and presence and typing indicators as best-effort features.

Non-functional requirements:

• Send-to-deliver latency of tens to low hundreds of milliseconds when both people are online.
• At-least-once delivery to devices, with exactly-once appearance in the UI through client deduplication.
• History kept for years, with heavy reads on recent messages.
• Order within a conversation, but not across conversations.

Estimates:

DAU:                 100M
Messages/user/day:   ~50  →  5B messages/day ≈ 60k msgs/s average (~200k peak)
Avg message size:    ~200 B stored metadata + text
Storage/year:        5B × 365 × 200 B ≈ 365 TB/year (before attachments)
Attachments:         object storage + CDN
Concurrent WS:       ~20–40M peak connections (not all DAU are online at once)

Check the arithmetic: 5 billion messages divided by 86,400 seconds is about 58,000 per second, which rounds to 60k. Then 5B × 365 × 200 B is about 365 TB.

Connection count, more than message rate, often sets the cost. Each open WebSocket holds memory and file descriptors on a gateway machine.

Common mistakes

• Starting with Kafka and Cassandra. Choose tools after you know the product, the fan-out pattern and the search needs.
• Sizing only by message rate. Open connections drive the gateway fleet, and they can dominate the bill.
• Demanding global order. Users need order within one conversation. Global order across all conversations is expensive and unnecessary.

Check yourself

1. Why is per-conversation order enough for chat? Users only notice order inside a conversation. Global order across all conversations costs far more and isn't needed.
2. If 100 million users each send 50 messages a day, how many messages per second on average? About 58,000, which the estimate rounds to 60,000.

Key takeaways

• Write down the requirements and the rough numbers before choosing any tool.
• Ask whether the product is end-to-end encrypted, and whether history must be searchable.
• Count open connections as well as messages per second.
• Order matters per conversation, not globally.

Further learning

Next, read about client message IDs, which explain how duplicate sends are avoided.