Traffic and messaging
Delivery semantics
The three promises a messaging system can make (at-most-once, at-least-once, exactly-once) and why the real choice is when the consumer says "done".
A consumer takes a message, starts work, and then crashes. Did the work happen? Was the message kept? Should it be sent again? Delivery semantics is the set of promises a messaging system makes about exactly these cases.
The big idea
Every promise depends on one moment: when the consumer tells the broker "I'm done". That signal is an ack (acknowledgement). Before the ack, the broker keeps the message. After the ack, the broker deletes it.
Concept
An ack is the consumer's message to the broker that says "this message is handled, you can delete it". It is sent after the work, not before.
There are three promises, and each one is a different answer to "when do we ack?"
| Promise | When is the ack sent? | If the consumer crashes mid-work | Good for |
|---|---|---|---|
| At-most-once | Before processing | Message is lost, never redone | Metrics, typing indicators |
| At-least-once | After processing | Message is redelivered, work may repeat | Almost everything else |
| Exactly-once | (Not achievable end to end) | (Not achievable end to end) | — |
Walking through a crash
Here is an at-least-once consumer that crashes after doing its work but before the ack.
The card was charged twice. The broker did exactly what it promised: it redelivered the message because it never got an ack. The bug is in the consumer, not the broker.
With at-most-once, the ack would have been sent first. The crash would then lose the message, and the customer would never be charged. Neither choice is free: one risks duplicates, the other risks losing work.
Why exactly-once is not a real option
"Do the work" and "record that it's done" are two separate steps, often in two separate systems. A crash can land between them, and no broker can know whether the work happened.
What you can build is effectively-once: at-least-once delivery plus an idempotent consumer. A duplicate then has no extra effect.
at-least-once delivery + idempotent handler = effectively-once
(duplicates happen) (duplicate is a no-op) (outcome happens once)Warning
Some products advertise "exactly-once" features. SQS FIFO, for example, drops messages with the same deduplication ID within a 5-minute window. It does not stop your consumer from processing a redelivered message after a crash. Read what the feature covers before you rely on it.
Common mistakes
- Assuming the broker prevents duplicates. It doesn't. Your consumer must.
- Acking before the work. This is the at-most-once trap. It looks fast, and it silently loses data during crashes.
- De-duplicating by message ID alone. Use a stable business key, such as
order_idplus action, or a row with a unique constraint.
Key takeaways
- Delivery semantics come down to when the consumer acks.
- At-least-once is the practical default, so duplicates are expected, not rare.
- Exactly-once across systems is not achievable. Build effectively-once from at-least-once plus idempotency.
Further learning
Next, see how brokers track unacked messages in acks and visibility timeouts. For the duplicate-handling side, read idempotency.
Related
Idempotency and retries
Making operations safe to retry, with idempotency keys, deduplicating consumers and jittered backoff, so at-least-once delivery never means a double charge.
Acks and visibility timeouts
How a broker knows a message is still being worked on, what happens when a worker disappears, and how to set the timeout so work isn't duplicated.
Message queues
A durable buffer between the code that creates work and the code that does it. Start here for the map of queue concepts.