Operating systems and concurrency
Shared state without locks
Designs that avoid shared mutable state, such as message passing, database constraints and fencing tokens, and why they hold up better across machines.
The safest lock is the one you don't need. If no two threads touch the same mutable data, there is nothing to race on. Many systems are built that way on purpose, and the ones that do share state can move the coordination to a place that is easier to get right.
Three approaches cover most cases: message passing within a process, database constraints for shared records, and fencing tokens across machines.
Message passing
In message passing, each piece of state has one owner. Other threads ask the owner to change it by sending a message, usually through a queue or channel. The owner processes messages one at a time, so the state needs no lock.
Go's channels and actor systems like Erlang's processes follow this pattern. The trade-offs are clear: messages copy data, which costs time, and a slow owner becomes a bottleneck. But the bugs of shared memory mostly disappear. A thread that sends a message can't corrupt the owner's state.
Message passing scales across machines too, because a queue can sit between services. The same idea shows up in queues.
Database constraints
Shared records, such as a username or a seat, belong in the database. A unique constraint lets the storage engine decide. Two concurrent inserts with the same key: one succeeds and one fails, with no application lock.
INSERT INTO seats (seat_id, user_id) VALUES ('14C', 'user_a'); -- succeeds
INSERT INTO seats (seat_id, user_id) VALUES ('14C', 'user_b'); -- fails: unique violationUse a conditional update for changes to existing rows, such as UPDATE ... WHERE version = 7. If the row changed, no rows match, and the app retries. This is optimistic concurrency. For multi-row invariants, use transactions, covered in transactions and isolation.
Fencing tokens
A lock held across machines is unreliable. A client can take a lease, pause for a long garbage collection, and wake up after the lease expired. Another client now holds the lock, and the first one writes anyway.
A fencing token fixes this. The lock service hands out a number that increases with each grant. Every write sends its token, and the storage system rejects writes with a token lower than one it has already seen. The stale client's write is refused, even though it still believes it holds the lock.
client A gets lock, token 33, pauses
lease expires; client B gets lock, token 34, writes
client A resumes, writes with token 33
storage: 33 < 34 already seen → rejectFencing needs the storage layer to check tokens. A lock service alone can't stop a stale writer. Kleppmann's essay on fencing, listed below, works through this carefully.
Note
Lock services don't remove the need for idempotent writes. A write that is retried after a timeout can still apply twice. See idempotency.
Common mistakes
- Sharing a mutable object across threads for convenience, then adding locks everywhere.
- Using a distributed lock without a fencing token, so a paused client can still corrupt state.
- Implementing uniqueness in application code with a check and then an insert, instead of a constraint.
Check yourself
- Why does a unique constraint avoid a race that a check-then-insert in code doesn't? Answer: the storage engine performs the check and the insert as one decision, so only one of two concurrent inserts succeeds.
- Why does a fencing token stop a paused client's write? Answer: the storage system has already seen a higher token, so the stale write's lower token is rejected.
Key takeaways
- Avoid shared mutable state where you can. Message passing gives each piece of state one owner.
- Put shared records in the database, and let unique constraints and conditional updates decide.
- Across machines, use fencing tokens, and check them at the storage layer.
- Make retried writes idempotent, because timeouts make duplicates likely.
Further learning
Martin Kleppmann's post How to do distributed locking explains fencing tokens with a worked example.
Next, return to the concurrency overview.
Related
Concurrency
Why shared state breaks when two threads touch it at once, and the tools that fix it, from locks and semaphores to atomics, message passing and database constraints.
Race conditions
A race condition happens when the result depends on the order in which threads run. How a read-modify-write loses an update, and why check-then-act bugs are so common.
Deadlock and livelock
Deadlock is threads waiting on each other forever. The four Coffman conditions that must all hold, the fixes that break one of them, and how livelock differs.
Idempotency and retries
Making operations safe to retry, with idempotency keys, deduplicating consumers and jittered backoff, so at-least-once delivery never means a double charge.