Distributed systems
Quorum reads and writes
With N replicas, a write succeeds after W of them acknowledge, and a read waits for R. When R + W is greater than N, every read overlaps every successful write in at least one replica.
A key-value store copies each key to N replicas. A write succeeds once W of them acknowledge. A read returns once R of them respond. You choose N, R and W, and that choice sets the trade-off between freshness, latency and availability.
Concept
Before you start. A coordinator is the node that receives a client request and talks to the replicas. A replica is one copy of a key. A quorum is the number of replicas that must agree for an operation to count.
The big idea
Think of a committee of N members. A decision needs W signatures. To check the latest decision, you must ask R members. If R + W is more than N, your group of askers must include at least one member who signed. That member has the latest decision.
This overlap is a pigeonhole argument. The picture breaks down when a write did not reach W replicas, or when two writers race each other.
Walking through it
With N = 3, these settings are common.
| Setting | Behavior |
|---|---|
W = 3, R = 1 | Fast reads, slow and fragile writes |
W = 1, R = 3 | Fast writes, expensive reads |
W = 2, R = 2 | Balanced default in many deployments |
W = 1, R = 1 | Maximum availability, more conflicts |
The overlap check is simple arithmetic.
W = 2, R = 2: 2 + 2 = 4 > 3 reads overlap every write
W = 1, R = 1: 1 + 1 = 2 <= 3 a read can miss the write entirelyHere is N = 3, W = 2, R = 2 in practice. A put goes to replicas A, B and C. A and B acknowledge, and the write returns. A later get asks two replicas. It hears B's new value and C's old value. The versions disagree, so the system repairs C.
Latency is set by the slowest replica you wait for. That is why R = N on a large cluster hurts reads.
Common mistakes
- Treating
R + W > Nas linearizable. Racing writers and sloppy quorums can still show stale values. See consistency. - Using
W = 1for important data. If that one node crashes before it copies the write, the data is gone. - Setting
R = Non a large cluster. The read waits for the slowest node every time.
Check yourself
- With
N = 5andW = 3, what is the smallestRthat guarantees overlap? 3. SinceR + Wmust exceed 5,Rmust be at least 3. - Why is
R = Ncostly? The read waits for the slowest of all replicas, so one slow node slows every read.
Key takeaways
Ncopies,Wacknowledgements for writes,Rresponses for reads.R + W > Nmakes every read overlap every successful write.- The overlap is a guarantee of freshness, not a proof of linearizability.
- Latency follows the slowest replica you wait for.
Further learning
Next, see what happens when the replicas you want are down, in sloppy quorums and hinted handoff.
For the same overlap idea applied to consensus, see majority quorums.
Related
Case study: distributed key-value store
Build a Dynamo-style store from first principles. Consistent hashing places the data, quorums set the consistency, vector clocks detect conflicts, and hints and Merkle trees repair failures.
CAP, PACELC and consistency models
What "consistent" means once data lives on several replicas, from linearizable to eventual, what each guarantee costs, and how to pick one per feature.
Majority quorums
A decision counts only when more than half the nodes accept it. Because any two majorities overlap, two conflicting decisions can never both win. This is why consensus clusters have 3 or 5 nodes.