Durable file writes

Say a service rewrites its config file, or saves a document. If it writes straight into the existing file and crashes halfway, the file holds half of the old contents and half of the new. That is a torn write, and the file may not even parse. The fix is an atomic replace: write the new version somewhere else, make it durable, then swap it into place in one step.

The swap is atomic because a rename within one file system either happens or doesn't. After a crash, readers see the old file or the new one, never a mix.

Walking through it

The pattern has four steps. Each one closes a gap that the previous step leaves open:

1. Write the new contents to a temporary file in the same directory.
2. fsync the temporary file, so its bytes are on disk before anything points at them.
3. rename the temporary file over the target. On POSIX systems, rename replaces the target atomically.
4. fsync the directory, so the rename itself survives a crash.

import os, json

def atomic_write(path: str, data: bytes) -> None:
    tmp = path + ".tmp"
    with open(tmp, "wb") as f:
        f.write(data)
        f.flush()
        os.fsync(f.fileno())          # step 2: data on disk before the rename
    os.replace(tmp, path)             # step 3: atomic swap on POSIX
    dfd = os.open(os.path.dirname(path) or ".", os.O_RDONLY)
    os.fsync(dfd)                     # step 4: persist the rename
    os.close(dfd)

atomic_write("config.json", json.dumps({"version": 2}).encode())
print(open("config.json").read())

Output:

{"version": 2}

Skipping step 2 is the most common mistake. The rename can reach disk before the data does, so after a crash the new name points at an empty or partial file. Skipping step 4 can lose the rename itself, and the old file comes back.

The temporary file must be on the same file system as the target. A rename across file systems isn't atomic, and it becomes a copy plus a delete.

Detecting torn writes

Atomic replace prevents torn files for updates you control. Some writes can't be made atomic. A log record may be written partly before a power cut, and a sector may be corrupted by the device. Here, store a checksum, such as CRC32 or a hash, with each record. On recovery, a record whose checksum doesn't match is treated as the end of the valid log, and everything after it is discarded.

This is how the write-ahead log knows where a crash cut off the tail. Without checksums, a partly written record could be replayed as if it were valid.

Common mistakes

• Writing directly into the live file and assuming a crash will leave it unchanged.
• Placing the temp file on another file system, then assuming the rename is atomic.
• Skipping the directory fsync, so a rename can roll back after a crash.
• Trusting a record without a checksum after a crash.

Check yourself

• Why does the rename happen only after the temporary file's fsync? Answer: otherwise the rename could reach disk first, leaving the name pointing at a file with no data yet.
• Why does a checksum help with a partly written log record? Answer: a torn record fails the checksum, so recovery can stop at the last complete record instead of replaying garbage.

Key takeaways

• An atomic replace writes a temp file, fsyncs it, renames it over the target, then fsyncs the directory.
• Keep the temp file on the same file system.
• Use checksums to detect torn records that atomic replace can't prevent.

Further learning

Next, see storage devices and SSDs for how the device underneath affects these guarantees.