Data building blocks
Cache layers
A request can be served from a cache at many layers, from the browser to the database. The closer a cache sits to the user, the more load it saves and the harder it is to invalidate.
A cache stores the result of expensive work, so the next request can skip the work. A hit means the answer was in the cache. A miss means the work ran. Caches don't live in one place. They sit at almost every layer between the user and the disk.
The big idea
Picture a stack of shelves between a customer and a warehouse. The shelf by the door is the fastest to reach, and it saves the most trips. It is also the hardest to restock, because you can't reach into every customer's home. The shelf in the back room is slower but easy to control.
Each layer trades speed against control. The browser cache is fastest, and you control it least. The database's buffer pool is slower to reach, and it manages itself.
Walking through it
| Layer | Example | Typical hit latency | Good for | How it's invalidated |
|---|---|---|---|---|
| Client | Browser cache, mobile app storage | About zero, no network | Static assets, rarely changing data | Hard. Use versioned URLs |
| CDN | Cloudflare, CloudFront, Fastly | 5–30 ms | Images, video, JS and CSS, public pages | Purge APIs and short TTLs |
| Reverse proxy | Nginx, Varnish | About 1 ms | Whole HTTP responses | Purge and TTL |
| In-process | An LRU map, Caffeine, lru-cache | About 100 ns | Tiny, very hot, rarely changing data | Per process. Copies drift |
| Distributed | Redis, Memcached, Valkey | 0.2–1 ms | Shared data across many servers | Explicit deletes and TTLs |
| Database | Buffer pool in Postgres or InnoDB | Inside the query | Hot pages | Automatic |
Most systems use several at once. A product page might come from the CDN for anonymous visitors. A logged-in request might read product data from Redis and fall back to Postgres, whose buffer pool keeps hot pages in RAM.
Common mistakes
- Forgetting which layer is serving stale data. When a user sees an old page, check the CDN, the browser and the proxy, in that order.
- Counting on client caches. You can't reach into a phone to delete a copy. Use versioned URLs, so a new version gets a new address.
- Trusting in-process caches across servers. Each process holds its own copy, so
Nservers can holdNversions.
Check yourself
- Why are in-process caches hard to keep consistent across servers? Each process has its own copy, so the copies drift apart whenever the data changes.
- Why do versioned URLs help with client caches? A new version gets a new URL, so devices fetch it fresh without you deleting anything on them.
Key takeaways
- A cache can sit at any layer, from the browser to the database buffer pool.
- The closer a cache is to the user, the more it saves and the harder it is to invalidate.
- Use several layers together, and know which one serves each request.
Further learning
The CDN and edge page covers the outer layer in depth.
Next, read about cache patterns, which decide how data gets into a cache.
Related
Caching
Where to put a cache, how to read and write through it, how to size it from hit-rate math, and how to keep it from serving stale data or melting down.
CDNs and the edge
Serve content from points of presence near users. A CDN cuts latency, absorbs attacks and cuts bandwidth costs, and its cache keys, headers and purge rules decide how well it works.
Cache patterns
A cache pattern decides who loads data into the cache and when writes reach the database. Cache-aside is the usual choice, because the cache is optional and the app still works without it.