Sloppy quorums and hinted handoff

A key's home replicas are its preference list, the nodes that should hold it. If one of them is down, a strict quorum write fails. Dynamo avoids that. It writes to a healthy node outside the list instead, and records a hint that names the intended owner. This is a sloppy quorum.

The big idea

Think of a neighbor who signs for your package while you are out. The package is not yours yet, but it is safe, and the neighbor hands it over when you come home. The arrangement keeps deliveries working during your absence.

The neighbor can lose the package before you return. That is why the design also needs a repair process, covered in Merkle anti-entropy.

Walking through it

Three replicas: A, B and C. C is down. N = 3, W = 2.

put(cart), preference list = [A, B, C], C is down
  A acknowledges              (1 of W = 2)
  D acknowledges with hint    (2 of W = 2)   D is not on the list; hint says "for C"
  put succeeds

C comes back:
  D sends the cart to C
  D deletes its hinted copy

The write succeeded even with one home node down. The hint is temporary. It lives on D, not on C, until C returns.

Here is the catch. A read may ask only A, B and C. D is not on the list, so it may not see the write. The overlap guarantee from R + W > N weakens. Sloppy quorums trade freshness for availability on purpose.

Common mistakes

• Treating a sloppy quorum as a strict one. It counts nodes outside the list, so overlap is not guaranteed.
• Treating hints as durable storage. If the hint holder crashes and loses the hint, nothing forwards the data. Anti-entropy must repair it.
• Expecting a read to always find the write. A read may never contact the stand-in node.

Check yourself

1. Does a sloppy write count if the stand-in node is not on the preference list? Yes. That is the point. The write counts toward W, even though the stand-in is outside the list.
2. What happens when the down node returns? The stand-in forwards the data to it, then removes its hinted copy.

Key takeaways

• A sloppy quorum writes to healthy nodes outside the preference list.
• A hint records the intended owner, so the data can be handed back.
• Writes succeed during failures, but reads may miss them.
• Hints are temporary. Anti-entropy covers what hints miss.

Further learning

Next, see how the system repairs keys that hints never delivered, in Merkle anti-entropy and read repair.

For the full design, see the distributed key-value store case study.