Arithmetic intensity and the ridge point

Peak TFLOPS on a datasheet are a ceiling, not a promise. Whether you can touch that ceiling depends on how much math you do per byte you move. That ratio is arithmetic intensity. The intensity where bandwidth and compute limits meet is the ridge point.

intensity I = FLOPs performed / bytes moved from HBM

The big idea

Every second a GPU can either move a fixed number of bytes or finish a fixed number of FLOPs. Whichever budget you empty first is the speed limit. Software that asks for the empty budget cannot go faster.

A kitchen that must fetch every spice from the basement for each pinch of salt is bandwidth-bound. One that stirs a full pot already on the stove is compute-bound. The picture breaks if the cook stands idle waiting for the next ticket — that is launch overhead, a third limit under both roofs.

Walking through the H100 ridge

The ridge is:

I_ridge = peak_FLOP/s / peak_bytes/s

For an H100 SXM with dense BF16 tensor-core peak ~989 TFLOPS and HBM ~3.35 TB/s:

peak_flops = 989e12
peak_bw = 3.35e12
print(round(peak_flops / peak_bw))

Output:

295

So I_ridge ≈ 295 FLOP/byte. Below that, attainable performance ≈ I × bandwidth. Above it, attainable performance ≈ peak_FLOP/s. FP8's higher peak raises the ridge further — you need even more reuse to saturate compute.

Bytes in the denominator are slippery. Academic rooflines often use DRAM counters. Napkin math for LLMs usually counts weight bytes streamed from HBM per decode step plus a rough activation term. Be consistent: when you compare two kernels, use the same byte model.

Common mistakes

• Using peak TFLOPS as expected speed. At low intensity you will never see that number.
• Mixing byte models. Counting L2 hits in one kernel and HBM in another makes the comparison meaningless.
• Forgetting writes. Write-allocate traffic on stores can make real intensity worse than the napkin.

Check yourself

1. What is I_ridge on this H100 BF16 sketch? Answer: about 295 FLOP/byte, which is 989 TFLOPS divided by 3.35 TB/s.
2. A kernel at 30 FLOP/byte on that GPU is limited by what? Answer: memory bandwidth; 30 is far below the 295 ridge.

Key takeaways

• Intensity is FLOPs per HBM byte. The ridge is peak FLOP/s ÷ peak bandwidth.
• Below the ridge you ride the bandwidth slope. Above it you hit the compute roof.
• Count bytes the same way every time you compare kernels.
• Decode napkins can start with weight plus KV traffic and still be useful.

Further learning

Next, place real kernels on that roof in roofline examples.