Realtime updates and long-running work

Two problems look different but share a fix. Clients want live updates without asking the server every few hundred milliseconds. And some work, such as encoding a video or settling a payout, cannot finish inside one HTTP request. For live updates, push data over a channel that stays open. For long work, run it as a job and let the client check on it.

The big idea

Think of a phone call versus a voicemail. For a live conversation, you need the line open, so updates arrive as they happen. For a task that takes time, you leave a ticket number and check its status later, or wait for a call back.

The analogy breaks when many clients are connected at once. Each open channel uses server resources, so a design needs to count them.

Walking through it

Choose the channel by the direction of the data.

Long work follows the job pattern. The client submits, gets a job ID, and checks the status.

POST /exports          -> 202 Accepted, job_id = j_881
worker picks up j_881  -> status: running
GET /jobs/j_881        -> status: running
worker finishes        -> status: done, result_url = ...

Polling is the cost to watch. A client polling every 500 ms makes 2 requests per second. A million such clients make 2 million requests per second, before any real work happens. Push the update instead, or make the poll interval longer.

Common mistakes

• Blocking a request on video encoding or a payout. The request times out, and the user sees an error for work that succeeds.
• Polling too fast across many clients. The poll load can exceed the real work.
• Accepting webhooks without deduplication. Providers resend events, so the handler must be safe to run twice.

Check yourself

1. A million clients each poll every 500 ms. How many requests per second is that? 2,000,000. Each client makes 2 requests per second.
2. When is SSE a better choice than WebSockets? When data flows only from server to client, such as a stream of tokens or availability updates.

Key takeaways

• Push live updates over WebSockets or SSE, not frequent polls.
• Run long work as a job with a status resource.
• Use webhooks for completion, and deduplicate them.
• Count open connections and poll rates when you size the system.

Further learning

For channel details and scaling, see realtime delivery.

For a payment flow that uses webhooks and idempotency, see the payment system case study.

The chat system case study uses WebSockets end to end.