GC pauses and tuning

Every garbage collector pays in three currencies: CPU time, pause length and memory headroom. Make one smaller and another gets bigger. Tuning means choosing which one your service can afford to lose, then measuring whether you got it.

The trade-off triangle

• Throughput: the fraction of CPU that goes to your code instead of the collector.
• Latency: how long the longest pauses last.
• Memory overhead: how much extra heap the collector needs to work efficiently.

You cannot get all three at their best. The JVM's Parallel GC maximizes throughput and accepts long pauses, which suits batch jobs. G1, the default since JDK 9, targets pause goals around 200 ms. ZGC and Shenandoah keep pauses under a millisecond on heaps of hundreds of gigabytes, at a cost in CPU and memory. Go targets sub-millisecond pauses and trades memory for CPU through one knob, GOGC.

Walking through it: how a pause becomes a p99 spike

Suppose a JVM service handles 1,000 requests per second, and G1 takes a 100 ms pause every 5 seconds:

Requests arriving during each pause:  1,000 req/s × 0.1 s = 100
Requests in each 5-second window:     1,000 req/s × 5 s  = 5,000
Fraction delayed by up to 100 ms:     100 / 5,000        = 2%

Two percent of requests are hit. The median can be 5 ms while the p99 includes the pause. The effect compounds in fan-out. If a page calls 20 backends, the chance that at least one is mid-pause is much higher than 2%.

The first check is the GC log. Use -Xlog:gc for the JVM and GODEBUG=gctrace=1 for Go, then line up pause times with latency spikes.

Knobs that matter

Go

• GOGC (default 100) sets how far the heap may grow past the live heap before the next collection: target ≈ live heap × (1 + GOGC/100). With 500 MB live, GOGC=100 lets the heap reach about 1 GB. GOGC=200 lets it reach 1.5 GB and halves the number of collections.
• GOMEMLIMIT (Go 1.19+) is a soft memory limit. The runtime collects more often as total memory nears it. In containers, set it to roughly 90% of the limit, for example GOMEMLIMIT=900MiB for a 1 GiB container.

JVM

• Choose the collector for the workload. G1 is balanced. ZGC (-XX:+UseZGC) suits latency-sensitive services. Parallel suits batch throughput.

A real case: Discord's Read States service

Discord's Read States service held a large in-memory LRU cache of read states, millions of entries per instance. The Go version showed latency and CPU spikes about every two minutes. The Go runtime forces a collection at least that often, and each collection scanned the whole cache. Shrinking the cache reduced the spikes but raised the miss rate and the database load.

Discord rewrote the service in Rust. Ownership frees an evicted entry at the moment of eviction, and nothing scans live data on a timer. The published 2020 comparison showed no periodic spikes and better tail latency.

The lesson is not that Go is bad. A tracing collector's cost scales with the live heap. A service that holds a huge, long-lived, pointer-rich structure is the worst case for it. Other options include moving the cache off-heap, or using pointer-free arrays that the collector does not scan.

Common mistakes

• Raising GOGC without watching RSS. A higher setting trades CPU for memory, and memory has a limit.
• Using ZGC for a batch job. Batch jobs usually want throughput, and lower pauses cost CPU.
• Reading average pause time. Look at p99 and the longest pause, because tail latency is what users feel.

Check yourself

• Why does a 100 ms pause every 5 seconds show up at p99 when the median looks fine? About 2% of requests arrive during a pause, so the tail includes the delay.
• What does GOGC=200 buy you, and what does it cost? Half as many collections, paid for with a heap that can grow to 1.5 times the live size.

Key takeaways

• Every collector trades throughput, pause length and memory headroom. You cannot minimize all three.
• A 100 ms pause every 5 seconds at 1,000 requests per second hits about 2% of requests, which lands in the p99.
• In Go, GOGC trades memory for CPU, and GOMEMLIMIT caps memory in containers.
• A tracing collector's cost grows with live data. Huge, long-lived caches are the hard case.

Further learning

Next, see how leaks and container memory limits cause a different kind of failure, in memory leaks and OOM kills.

HotSpot Virtual Machine Garbage Collection Tuning Guide (JDK 21) is the reference for JVM collector choices.