Traffic and messaging
Synchronous vs asynchronous work
The difference between waiting for a task to finish and handing it off, and why that choice shapes latency, failures and user experience.
When someone clicks "Place order", the server has several jobs to do: charge the card, reserve stock, send a confirmation email, and update analytics. The key question is which jobs must finish before the server replies.
That question splits all work into two kinds:
- Synchronous: the caller waits until the work is done, then gets the result in the same request.
- Asynchronous: the caller hands the work off, gets back "received", and the work happens later.
The big idea
Think of a restaurant kitchen.
- Synchronous: the waiter stands at the kitchen door until your steak is cooked, then brings it to you.
- Asynchronous: the waiter pins a ticket on the rail and goes to serve other tables. The cooks work through the tickets.
Neither is better. Synchronous is simpler, and you know the result immediately. Asynchronous frees the waiter, but now someone has to track whether the steak was cooked.
Concept
A request is a message from a client to a server that expects a reply. An acknowledgement (ack) is a short reply that means "I received this", which is not the same as "I finished this".
Walking through the timing
Here is a checkout that does everything synchronously.
Synchronous checkout (the user waits for all four steps)
charge card 400 ms needed to know the order succeeded
reserve stock 50 ms needed to know the order succeeded
send email 800 ms not needed to show "order placed"
update analytics 200 ms not needed to show "order placed"
total 1450 msThe last two steps don't change what the user sees, but they add 1 second. Now hand them off.
Asynchronous checkout
charge card 400 ms
reserve stock 50 ms
queue email + analytics 5 ms just write two messages
total 455 msThe page loads in about a third of the time.
There is a reliability gain too. If the email provider is down for 30 seconds, the synchronous checkout fails for every customer. The asynchronous one succeeds, and the email is sent once the provider recovers.
The cost of async
Asynchronous work is not free. You now have to handle three things:
- Unknown completion. The caller doesn't know when, or whether, the work finished. You need a status field or a notification.
- Lost hand-offs. If the server crashes after the card is charged but before the email message is written, the email is never sent. The fix is covered in message queues (the outbox pattern).
- Duplicates. Retries can run the same work twice, so the work must be idempotent.
Trade-off
Use synchronous when the caller needs the answer to continue, such as checking a password or charging a card. Use asynchronous when the response does not depend on the result, such as sending an email, making a thumbnail or updating analytics.
Common mistakes
- Making the user-visible step async. Replying "payment succeeded" before the card is charged is a lie that breaks trust.
- Treating "accepted" as "done". A
202 Acceptedresponse means the work is queued, not finished. Show "processing" in the UI. - Doing slow work synchronously by habit. If a request handler waits on a thumbnail resize, every upload is as slow as the resize.
Check your understanding
- A user uploads a photo and you must create three thumbnail sizes. Should that be sync or async? Async, because the user doesn't need thumbnails to see "upload complete".
- Why does an async consumer still need idempotency? Because a retry after a crash can run the same job twice.
Key takeaways
- Synchronous work: the caller waits for the result. Asynchronous work: the caller gets an acknowledgement and the result comes later.
- Move work off the request path when the response doesn't depend on it.
- Async makes the system faster and more resilient, but you must track completion, handle lost hand-offs and tolerate duplicates.
Further learning
Continue with message queues, the most common tool for async work, and then idempotency for handling duplicates.
Related
Message queues
A durable buffer between the code that creates work and the code that does it. Start here for the map of queue concepts.
Availability, latency and the tail
Nines, percentiles and the math of composing services. Why p99 matters more than the average, how fan-out amplifies the tail, and how to design for both uptime and speed.
Idempotency and retries
Making operations safe to retry, with idempotency keys, deduplicating consumers and jittered backoff, so at-least-once delivery never means a double charge.