Distributed systems
Last-write-wins and clock skew
Last-write-wins keeps whichever write carries the higher timestamp. When clocks disagree, that is not the write that happened last, and the loss is silent.
Many distributed databases settle a conflict with last-write-wins (LWW). Each write carries a timestamp. When two writes to the same key meet, the one with the higher timestamp survives. It's simple to build and easy to explain. It also silently discards writes whenever clocks disagree.
Concept
Clock skew is the difference between two machines' clocks at the same real moment. Before you read on, see clock drift and NTP, which explains how skew arises.
The big idea
LWW keeps the write with the larger timestamp. It does not keep the write that happened later. Those are the same only when every clock is right.
Picture two people correcting one whiteboard, each with a different watch. The person whose watch runs ahead always wins, even when they wrote first. The mistake is invisible, because both people saw a successful save.
Walking through it
Node A's clock runs 50 ms ahead of real time. Node B's clock is accurate.
real time 10:00:00.000 client 1 writes status="shipped" via node A
node A stamps it 10:00:00.050
real time 10:00:00.020 client 2 writes status="cancelled" via node B
node B stamps it 10:00:00.020The cancellation really happened 20 ms after the shipping update, so it should win. But 0.050 > 0.020, so LWW keeps "shipped" and discards "cancelled." Both clients got a success response. No error appears in any log. The order ships.
The loss grows with the skew. Any write that lands within the skew window of a conflicting write is a coin flip. If a misconfigured node's clock is an hour ahead, its writes beat every other node's writes for an hour.
LWW is fine when losing a concurrent update is harmless, such as a cache entry or a "last seen" timestamp. It's dangerous for orders, balances and carts. The conflict resolution page covers the alternatives.
Common mistakes
- Using LWW for money or orders. A lost update there means a lost payment or a lost cancellation.
- Assuming the later write wins. LWW keeps the larger timestamp, which only matches real order when clocks agree.
- Trusting that a successful response means the write survived. Both writes can succeed, and one still disappears.
Check yourself
- In the example, why does "shipped" survive? Node A's clock is 50 ms ahead, so its earlier write carries a larger timestamp.
- When is LWW acceptable? When losing a concurrent update is harmless, such as a cache entry or a last-seen time.
Key takeaways
- LWW keeps the larger timestamp, which isn't always the later write.
- Clock skew can make LWW silently discard the write that really came last.
- Use LWW only where losing a concurrent update is harmless.
- For data that matters, use a conflict-aware method, such as version vectors.
Further learning
Next, read about Lamport clocks, which order events without trusting any wall clock.
Related
Time, clocks and ordering
Why wall clocks lie across machines, how skew silently loses writes, and how Lamport clocks, vector clocks, hybrid logical clocks and TrueTime order events you can't time directly.
Conflict resolution and CRDTs
What to do when replicas accept concurrent writes. The options run from last-write-wins and siblings to CRDTs, operational transformation and the sync engines used by collaborative editors.
Clock drift and NTP
Every server's clock drifts, and NTP only corrects it within the limits of the network path. A clock is an estimate with an error bar, not an exact reading.