Cache failure modes

Beyond stampedes, a cache can break in three other ways. The load lands in the wrong place each time. One key overwhelms one node. Missing keys keep reaching the database. An empty cache sends every read to a database sized for a much higher hit rate. Each one has its own fix.

Hot keys

A hot key is one key that gets a large share of traffic. Say a celebrity's profile gets 200,000 reads per second. A single Redis process typically handles on the order of 100,000 to 200,000 simple operations per second, depending on payload size and pipelining. The node that owns the key hits its limit.

Three fixes:

• Local in-process cache for the hottest keys, with a TTL of 1 to 5 seconds. If 200 app servers each read the key once per second, Redis sees 200 requests per second, not 200,000.
• Key replication. Store copies such as profile:42#0 through profile:42#7 on different nodes, and read one at random. Writes must update or delete every copy.
• Read replicas for the Redis shard that holds the key.

Penetration

Cache penetration happens when requests ask for keys that don't exist. A cache never holds a missing key, so every request reaches the database. The cause can be a client bug looping on a deleted ID, or an attacker sending random IDs.

• Negative caching. Store "not found" with a short TTL, such as 30 seconds. Repeated requests for the missing ID stop reaching the database.
• A Bloom filter in front. Load every valid ID into the filter. A "definitely not present" answer skips the cache and the database.

Size of a filter: a 1% false-positive rate needs about 9.6 bits per element. For 100 million IDs, that's about 120 MB.

100,000,000 IDs × 9.6 bits = 960,000,000 bits ÷ 8 = 120,000,000 bytes ≈ 120 MB

Cold starts

After a deploy changes key formats, a cache flush, or a failover to an empty replica, the hit rate drops to 0%. The database gets the full read load. If it was sized for a 1% miss rate, it can fall over.

Mitigations:

• Warm the cache from a snapshot, or replay recent keys, before you take traffic.
• Roll out key-format changes gradually.
• Use Redis replication, so a failover keeps the data.
• Rate-limit misses, so the database slows down instead of collapsing.

Common mistakes

• Caching only the hits. Misses on missing keys stay expensive. Cache the "not found" answer too.
• Replicating hot keys without updating every copy. Stale copies persist until their TTL.
• Flushing the whole cache on deploy. You trigger a cold start on purpose. Change keys gradually.

Check yourself

1. Why does local caching for a hot key cut Redis traffic so much? Each app server asks Redis once per TTL interval instead of once per request, so 200 servers send about 200 requests per second.
2. Why cache "not found" results? So repeated requests for a missing ID stop reaching the database each time.

Key takeaways

• Hot keys need local caches, replicated copies or read replicas.
• Penetration needs negative caching, and a Bloom filter for large ID spaces.
• Cold starts need warm-up, gradual rollouts and rate-limited misses.

Further learning

The probabilistic data structures page explains Bloom filters and their relatives in more depth.

Next, read about the memcache case study, which shows how one large operator handled these problems.