The transactional outbox

Many sagas, especially choreographed ones, need a service to update its database and publish an event as one step. That is a dual write again. Write to the database and then publish, and a crash loses the event. Publish first, and you announce a change that may never commit. The transactional outbox removes the problem with one trick: write the event into the same database, in the same local transaction as the change.

Writing the event with the change

BEGIN;
INSERT INTO orders (id, user_id, total, status)
  VALUES ('o_123', 'u_9', 4000, 'PLACED');
INSERT INTO outbox (id, aggregate_id, type, payload)
  VALUES ('evt_456', 'o_123', 'OrderPlaced',
          '{"orderId":"o_123","total":4000}');
COMMIT;

Either both rows exist, or neither does. The event can't exist without the change, and the change can't commit without its event.

Getting rows to the broker

Something must move outbox rows to the broker. There are two common ways:

• Polling publisher. A background job runs SELECT ... FROM outbox WHERE published_at IS NULL ORDER BY id LIMIT 100, publishes the rows and marks them published. It is simple. It adds database load, and latency equal to the polling interval.
• Change data capture (CDC). A tool such as Debezium tails the database's replication log, the Postgres WAL or the MySQL binlog, and turns each new outbox row into a Kafka message. Latency is lower, there are no polling queries, and Debezium ships an outbox event router for this pattern. See change data capture.

The guarantee is at-least-once

The relay can crash after it publishes but before it marks the row. On restart, it publishes the row again. So the outbox gives you at-least-once delivery, not exactly-once. That is fine, as long as the consumers handle duplicates.

The inbox: deduplicating on the consumer

The consumer side mirrors the outbox. It records each processed event ID in the same transaction as its side effects:

BEGIN;
INSERT INTO inbox (event_id) VALUES ('evt_456');   -- PRIMARY KEY on event_id
-- If that insert hits a unique violation, the event was already handled:
-- roll back, acknowledge the message, and stop.
UPDATE stock SET reserved = reserved + 1 WHERE sku = 'sku_77';
COMMIT;

A duplicate delivery fails the unique constraint and changes nothing. Outbox on the producer plus inbox on the consumer gives effectively-once processing across services, with no distributed transaction. This combination underpins most reliable event-driven systems and the payment system case study.

Common mistakes

• Publishing from the request path after commit. That reintroduces the dual write. Let the relay publish instead.
• Using an outbox without an inbox. At-least-once delivery then creates real duplicates.
• Forgetting the relay is a moving part. It needs monitoring. If it stops, events quietly pile up.

Check yourself

1. Why does the outbox row go in the same transaction as the business change? Both commit or neither does. The event can't exist without its change, and the change can't commit without its event.

2. The relay publishes an event and crashes before marking it. What happens? The row is published again after restart, so consumers must deduplicate, usually with an inbox.

Key takeaways

• Write the event in the same database transaction as the change.
• A polling relay is simple. CDC is lower latency and avoids polling queries.
• The outbox gives at-least-once delivery.
• An inbox on each consumer turns that into effectively-once processing.

Further learning

Debezium's write-up walks through the outbox pattern with CDC in practice.

Reliable microservices data exchange with the outbox pattern (Debezium)Debezium is an open source distributed platform for change data capture. Start it up, point it at your databases, and your apps can start responding to all of the inserts, updates, and deletes that other apps commit t…Debezium

Next, see how to handle scarce resources with holds in reservations and try-confirm-cancel.