Machines and programs
Memory management and garbage collection
How languages decide where values live and when memory is given back, the trade-offs of manual, ownership, reference-counted and tracing approaches, and how to find leaks and size heaps safely.
Every value your program creates has to live somewhere. Eventually that space must be given back. How a language answers "where?" and "when is it given back?" shapes its speed, its safety and its failure modes.
C makes you free memory by hand, and mistakes become security bugs. Rust makes the compiler check your work. Python counts references. Go, Java and JavaScript run a garbage collector that finds dead objects for you, and that can make your p99 latency jump while it runs.
You do not need to write a garbage collector to be a good backend engineer. You do need to know why a Java pod with -Xmx4g gets OOM-killed at a 4 GB limit, why a Go service's latency spikes on a regular rhythm, and how to find a leak that grows memory by 200 MB a day.
Concepts in this section
Start with the first page. The rest assume it.
| Step | Concept | What it answers |
|---|---|---|
| 1 | Stack, heap and allocation | Where does a value live, and what does allocating it cost? |
| 2 | Four ways to free memory | Who frees memory, and at what risk? |
| 3 | How tracing garbage collectors work | How does a GC find the garbage? |
| 4 | GC pauses and tuning | Why do latency spikes appear, and what can you change? |
| 5 | Memory leaks and OOM kills | Why does memory grow, and why does a process disappear? |
Why it matters
Memory decisions show up in three places. They show up as tail latency when a collector pauses your program. They show up as cost when a service needs bigger instances than its data justifies. They show up as outages when a container is killed for using more memory than its limit. Each one has a clear diagnostic path, covered in the pages above.
Common mistakes
- Sizing a JVM heap to the container limit. Non-heap memory such as thread stacks and metaspace also counts toward the limit.
- Trusting "the heap is only 4 GB". Process memory is larger than the heap.
- Tuning GC by folklore. Change one knob, then compare pause times and RSS under load.
- Calling every slow service a GC problem. Most GC problems are allocation problems, and the cheapest fix is allocating less.
Key takeaways
- The stack is fast and scoped to a call. The heap holds anything that outlives a call. Escape analysis decides which one a value gets in Go and the JVM.
- The four approaches (manual, ownership, reference counting, tracing) trade safety, predictability and overhead.
- Tracing collectors pay in memory headroom, CPU and pauses. Their cost depends on how much live data they must scan.
- Leaks in garbage-collected languages are objects that are still reachable and never used again. Diff heap profiles to find them.
Further learning
Computerphile gives a short, clear visual explanation of mark-and-sweep. Watch it first if tracing GC is new to you.
Operating Systems: Three Easy Pieces covers memory management from the operating system side.
Related
The memory hierarchy
From registers to SSDs to the network, the latency numbers that dominate performance, and why the tier your data sits in matters more than the code you write.
Processes, threads and the kernel
What the OS gives every program, and how to see it on a live Linux box. Start here for the process model.
Kubernetes in production
Requests and limits, probes, autoscaling, disruption budgets, packaging and the mistakes that cause outages.