Cloud storage and RAID

A single disk fails eventually. RAID (redundant array of independent disks) combines several disks so that one or more can fail without losing data. Cloud providers offer the same idea at larger scale, under different names. Each option makes different promises about durability, speed and what happens when the hardware breaks.

RAID levels

Rebuild time is the hidden cost. A 20 TB disk rebuilt at 200 MB/s takes about 28 hours, because the rebuild must read every surviving block:

20 TB / 200 MB/s = 100,000 s ≈ 27.8 hours

During those hours, a second failure can lose the array in RAID 5. Large disks make RAID 5 risky for that reason, and RAID 6 or mirrored layouts are safer.

Erasure coding

Large object stores use erasure coding instead of RAID. Data is split into k data fragments, and m parity fragments are added, so any k of the k + m fragments can rebuild the object. Spread across different machines, racks or zones, the fragments survive the loss of several of them. A common layout stores 1.5 times the data, not the 3 times that three full copies need.

The trade-off is more machines touched per read and write, and more network traffic during repair.

What cloud volumes promise

Cloud block storage and instance stores differ on one question: does the data survive when the machine goes away?

• Network block volumes (such as EBS or persistent disks) are replicated within a zone by the provider. The volume survives an instance stop or failure, and you can attach it to another instance. Their performance is limited by the network and by the volume tier you pay for.
• Instance-store volumes are local SSDs on the host. They are fast, but the data disappears when the instance stops or the host fails. Use them for caches and scratch space, not the only copy of anything.
• Object storage (such as S3) stores whole objects behind an HTTP API, with erasure coding across zones. It is durable and cheap, but it isn't a file system. Writes replace whole objects, and small random updates are slow.

Durability figures describe the storage system, not your application. A bug that deletes the wrong object is still your bug, and replication copies mistakes too. Keep versioning and tested restores.

Common mistakes

• Choosing RAID 5 on very large disks, where a rebuild takes long enough for a second failure.
• Treating RAID as a backup. It protects against disk failure, not deletion or corruption.
• Putting the only copy of important data on an instance store.
• Using object storage for a database's random small writes.

Check yourself

• Why is a rebuild of a large RAID 5 array risky? Answer: the rebuild takes many hours while the array has no redundancy left, so a second disk failure during it loses the data.
• Why does an instance store lose data that a network volume keeps? Answer: the instance store is local to one host, so when the instance stops or the host fails, its disks go with it.

Key takeaways

• RAID trades capacity for redundancy, and rebuild time is the hidden risk.
• Erasure coding gives similar protection across machines at lower overhead, with more network cost.
• Network volumes survive instance failure, instance stores don't, and object storage isn't a file system.
• RAID and replication aren't backups. Keep versions and tested restores.

Further learning

Next, return to the files and storage overview, or see blob storage for object storage design.