Files and storage

A file looks simple: a name, some bytes, and a write call that returns. Underneath, the operating system splits the work across several layers. A name points to an inode. Writes sit in memory in the page cache. A journal records changes, and the device stores them in its own way, with its own write rules and wear.

Most data-loss bugs happen in the gaps between these layers. A write returns, but the bytes are still in memory. A rename is durable, but the file it replaced was not. Knowing the layers tells you which call actually promises durability.

The big idea

Three layers decide what survives a crash:

1. The file system maps names to inodes and keeps metadata consistent through a journal.
2. The page cache holds file data in RAM. A plain write only reaches this layer.
3. The device writes to flash or magnetic media, and it may cache writes too.

Durability requires an explicit flush through all three, using fsync, and that flush can fail in ways your code must handle.

Concepts in this topic

Why this matters

Databases, message brokers and log systems all need durable writes, and their correctness depends on the file system layer. The write-ahead log assumes that a flushed record stays flushed. Object storage, covered in blob storage, makes different promises, and those differ from a local disk.

Which layer to suspect

When data is missing after a restart, check the layers in order. Did the application call fsync, or only write? Did it flush the directory after a create or rename? Does the device honor flushes, or does it hold a volatile write cache without protection? Did a RAID rebuild or a volume failover lose the only copy? Each question rules out one layer, and the first "no" usually points at the bug.

Common mistakes

• Treating a successful write call as durable.
• Ignoring the result of fsync, which can fail even when the write seemed to succeed.
• Assuming a cloud volume has the same guarantees as a local disk, or as object storage.

Key takeaways

• A write returns when the data reaches the page cache, not the disk.
• fsync is the durability point, and its errors must be handled.
• Devices add their own caches, wear and write amplification to the story.

Further learning

Next, see file systems and journaling for how names and inodes are organized.