Distributed transactions

Inside one database, a transaction is a gift. You write BEGIN, update three tables and COMMIT. The database guarantees that all three changes happen or none do, even if the server loses power halfway. That guarantee ends when one business operation spans two databases, or two services that each own a database. Placing an order might mean writing the order, charging a card and reserving stock in three separate systems. Any of those steps can fail after the others have succeeded.

This topic covers the tools for that situation, from strongest and most expensive to weakest and most practical. It also covers the most useful advice: often the best distributed transaction is the one you avoid by drawing boundaries differently. You should know transactions and isolation and failure models first.

Reading order

Read the first two in order. The rest depends on them.

Choosing an approach

Redraw the boundaries first

The most effective distributed transaction is the one you don't need. Before you reach for a saga, ask why the data that must change together lives in different places.

• Things that must always agree probably belong together. If orders and order_line_items live in separate services, and a saga keeps them in sync, that's a boundary mistake. See monoliths and microservices.
• Design around invariants. Pat Helland's "Life beyond Distributed Transactions" argues that each unit of transactional consistency should sit on one machine, with messages between units. Cross-unit work becomes asynchronous.
• Ask whether the invariant must be immediate. "Never sell more seats than exist" must be. "The order count on a profile matches the orders" can lag by seconds.

Mistakes that cause the most damage

• Splitting data that must agree. Then you need a saga for something a single transaction would have handled.
• Assuming a saga is isolated. Other reads can see intermediate states.
• Calling a timeout a failure. It is an unknown outcome, and a retry needs an idempotent step.

Key takeaways

• When one operation writes to two systems without a shared transaction, a crash between the writes leaves them inconsistent.
• Two-phase commit gives atomicity across participants, but it blocks if the coordinator fails after prepare.
• Distributed SQL makes 2PC practical by replicating the coordinator with consensus, at the cost of latency.
• Sagas commit local steps in sequence and undo them with compensations. They are atomic eventually, but not isolated.
• The outbox publishes events atomically with database writes. An inbox on the consumer makes processing effectively once.
• The best fix is often to redraw boundaries so data that must change together lives together.

In practice

• Spot dual writes in code review. Ask what happens if the process dies between two writes.
• Make saga state visible. A saga_instances table or a workflow engine's UI shows where a stuck saga is.
• Build reconciliation anyway. A nightly job that compares orders, payments and shipments catches what the protocols missed.
• In interviews, name the pattern and the reason, such as "checkout is an orchestrated saga, with events sent through an outbox, and each step idempotent on the order ID."

Further learning

Pat Helland's essay argues for designing around entities rather than cross-system transactions.

Life beyond Distributed Transactions (Pat Helland, ACM Queue)queue.acm.org Designing Data-Intensive Applications (Kleppmann)Designing Data-Intensive Applications (DDIA) — an O’Reilly book by Martin Kleppmann (The Wild Boar Book)dataintensive.net

Start with the dual-write problem.