Hit rate and sizing

A cache is worth exactly as much as the traffic it absorbs. The hit rate is the share of reads it answers. The rest, the miss rate, reaches the database. That one number decides both the latency users see and the load on your database.

The big idea

Hit rate looks like a latency metric, but it is really a capacity metric. Small changes near 100% change the database's load a lot. Going from 90% to 99% hits sounds like a small gain. It actually cuts the misses from 10% to 1%, a tenfold reduction.

So when your database struggles, check the hit rate before you add hardware. A cache that dropped from 99% to 90% has quietly multiplied your database's work.

Walking through it

Two formulas cover most of the math.

effective latency = hit% × cache latency + miss% × (cache latency + DB latency)
database load     = request rate × miss%

Work through an API that serves 50,000 reads per second. A Redis hit costs 0.5 ms. A miss costs the 0.5 ms check plus a 10 ms database query.

Hit rate 90%:
  latency = 0.90 × 0.5 + 0.10 × 10.5 = 0.45 + 1.05 = 1.50 ms
  DB load = 50,000 × 0.10 = 5,000 queries/s

Hit rate 99%:
  latency = 0.99 × 0.5 + 0.01 × 10.5 = 0.495 + 0.105 = 0.60 ms
  DB load = 50,000 × 0.01 = 500 queries/s

If your database is sized for 500 queries per second, a drop to 90% hits overloads it tenfold.

Sizing. Size for the working set. Suppose 100 million users each have a 1 KB profile, which is 100 GB in total. But 20% of users cause 90% of reads.

Hot set = 100M × 20% × 1 KB = 20 GB
Add ~30% for Redis per-key overhead → 26 GB
Leave headroom for growth and replicas → 3 shards × 16 GB, or 2 × 32 GB

Then measure. A low hit rate with high eviction means the working set doesn't fit. A high hit rate with half-empty memory means you over-provisioned.

Eviction. When memory fills, something must go.

Modern algorithms such as W-TinyLFU, used in Caffeine, and S3-FIFO combine recency and frequency. They beat plain LRU on most real traces.

Common mistakes

• Sizing for total data. Measure the working set. Most of the data is never read in a given window.
• Judging a cache by latency alone. Track hit rate, because it predicts database load.
• Ignoring scans. A batch job that touches every key can evict the hot data your users need.

Check yourself

1. The database handles 500 queries per second. Traffic is 50,000 reads per second. Hit rate falls from 99% to 90%. What is the new database load? 5,000 queries per second, ten times the old load.
2. Why does plain LRU suffer from scans? A scan touches many keys once, and those keys push out data that was used recently but not in the last moment.

Key takeaways

• Hit rate determines database load, so treat it as a capacity metric.
• Going from 90% to 99% hits cuts misses tenfold.
• Size for the working set, then confirm with hit rate and eviction counts.
• Pick an eviction policy that resists scans, such as LFU or a modern algorithm.

Further learning

The estimation page covers the arithmetic behind QPS and storage sizing.

Next, read about Redis specifics, the cache most teams use in practice.