Stack, heap and allocation

A running thread keeps data in two main places. The stack holds data tied to one function call. The heap holds data whose lifetime is not tied to a single call, such as a user object returned from a function or a cache shared across requests. Knowing which one a value lives in tells you when it dies and what it costs.

The big idea

Each active function call gets a frame on the stack. The frame holds its local variables, arguments and return address. Calling a function moves the stack pointer down by a fixed amount. Returning moves it back. That is a single register addition, almost free, and the top of the stack is usually hot in L1 cache.

The catch is that a frame's data dies when the function returns. The stack is also small: about 8 MB by default for the main thread on Linux, about 1 MB for a Java thread, and only a few KB for a new goroutine, which grows on demand.

The heap is a larger pool. Allocating from it means asking an allocator, such as malloc or the runtime, for a free block of the right size. Someone eventually has to free that block.

A garbage collector finds heap data that nothing can reach any more. The stack frames and global variables are the roots. Anything reachable from a root by following pointers is live. The rest is garbage.

Walking through it: escape analysis

In Go and Java you do not choose stack or heap. The compiler does, using escape analysis. If a value's address never leaves the function, the value can stay on the stack. If it escapes, for example by being returned or stored in a global, it goes on the heap.

type Point struct{ X, Y int }

func sum(a, b int) int {
	p := Point{a, b} // stays on the stack: never escapes
	return p.X + p.Y
}

func newPoint(a, b int) *Point {
	p := Point{a, b} // escapes: its address outlives the call
	return &p
}

Go will report its decisions:

$ go build -gcflags=-m ./...
./point.go:9:2: moved to heap: p

The JVM's C2 compiler does something similar at run time. It can break a non-escaping object into plain local variables, so the object is never allocated at all.

Why allocating in a hot loop hurts

Heap allocation is not slow in itself. A JVM thread allocates by bumping a pointer in a thread-local buffer, which is about as cheap as a stack push. Go handles small objects in tens of nanoseconds. The real cost comes later. Every allocated byte brings the next collection closer, and each collection does work in proportion to the live heap. Allocation also spreads data around memory, which hurts cache locality.

Building a string in a loop shows the problem:

// Allocates a new, longer string on every iteration
s := ""
for _, part := range parts {
	s += part + ","
}

// One growing buffer
var b strings.Builder
b.Grow(estimatedSize)
for _, part := range parts {
	b.WriteString(part)
	b.WriteByte(',')
}
s := b.String()

With 10,000 parts of 10 bytes each, the first version copies about 11 + 22 + … + 110,000 ≈ 10,000² × 11 / 2 ≈ 550 MB and makes 10,000 allocations. The second copies about 110 KB with a handful of allocations.

Go benchmarks with b.ReportAllocs() show the allocation count directly, which is often a better target than raw nanoseconds. Other fixes are reusing buffers (sync.Pool in Go), pre-sizing slices and maps, and avoiding boxing. A List<Integer> in Java stores a heap object per element. An int[] stores raw 4-byte values.

Common mistakes

• Assuming heap allocation is always slow. It is often cheap. The cost is the garbage it creates later.
• Building strings with += in a loop. Each step copies the whole string so far.
• Keeping a small slice of a big buffer alive. In Go, a 10-byte sub-slice keeps the whole backing array alive. Copy the bytes you need.

Check yourself

• Why does the newPoint function allocate on the heap while sum does not? newPoint returns the address of its local, so the value outlives the call and escapes.
• Why is the += loop expensive for 10,000 parts? Each iteration copies the whole string built so far, which adds up to about 550 MB of copies.

Key takeaways

• The stack holds data for one call and is almost free to use. The heap holds data that outlives the call.
• Escape analysis in Go and the JVM decides which one a value gets. go build -gcflags=-m shows the decisions.
• Allocation is cheap at the moment it happens. The cost arrives later as garbage collection work.
• Reduce allocations in hot paths: reuse buffers, pre-size collections and avoid needless boxing.

Further learning

Next, see the four ways a language can free memory in four ways to free memory.

Understanding Ownership (The Rust Programming Language) explains how Rust decides where values are freed.