Metadata and object consistency

Object storage holds the bytes. A database holds everything you query. A files row stores the owner, the object key, the size, the checksum and the status. The bytes live elsewhere. The two systems share no transaction, so they can disagree.

The big idea

Two bad states can appear.

• An object with no row. The upload finished, but the server crashed before recording it. You keep paying for bytes nobody can find.
• A row with no object. The row was written, but the upload failed or never finished. Links to it break.

Think of a coat check. The ticket is the row, and the coat is the object. A ticket without a coat, or a coat without a ticket, is a mess. The fix is a protocol that makes each step recoverable.

Walking through it

A robust protocol has five steps.

1. Write intent first. Insert the row as PENDING, with a server-chosen key, before handing out the upload URL.
2. Upload the bytes directly to the store.
3. Confirm. Mark the row AVAILABLE when the client calls "complete." Better still, also use the store's event notification when the object appears. Both paths make the update idempotent, so a client that disappears is covered.
4. Garbage-collect. A periodic job deletes PENDING rows older than a day, along with their objects. A slower job compares the bucket with the database, using a listing or an inventory report, and deletes objects with no row.
5. Never overwrite a key. Give every version a new key, such as a UUID or a content hash. Readers and caches never see a half-replaced object, and you never need to purge.

Deletes run in reverse. Mark the row deleted first, so users see the file vanish at once. Delete the object later, after a grace period that allows undo.

Consistency also matters. Since December 2020, S3 gives strong read-after-write consistency for PUT, DELETE and LIST. Before that, overwrites and listings could be stale. Some older designs and third-party stores still assume that. GCS and Azure Blob are also strongly consistent for object operations.

Common mistakes

• Writing the row only after the upload. A crash in between leaves an orphan with no record.
• Skipping garbage collection. Orphans accumulate and bill you forever.
• Overwriting keys. A reader can see half of the old object and half of the new one, and caches keep the old copy.

Check yourself

1. Why insert a PENDING row before the upload? If the client never finishes, a cleanup job can find the stale row and delete its object.
2. Why never overwrite a key? Readers and caches never see a half-replaced object, and you don't need to purge anything.

Key takeaways

• The store holds bytes, the database holds facts about them, and no transaction joins them.
• Write intent first, confirm after the upload, and clean up what crashes leave behind.
• Use a new key for every version.
• Deletes mark the row first, then remove the object after a grace period.

Further learning

The S3 retrospective from AWS explains how the team built and runs the store at large scale.

Next, read about storage classes and egress cost, which covers what all this storage costs.