"Decode is memory-bound" is the most repeated claim in LLM inference and almost nobody attaches a number to it. Memory-bound against which ceiling, on which card, at what batch size? The claim is usually made about a datacentre GPU in FP8 and then applied to whatever hardware is on hand.
Both ceilings measured on the card itself, then a real decode step decomposed against them. At batch 1 the answer is neither: the largest slice is not kernel work at all, and it disappears the moment you batch.
Every verdict further down is relative to these two lines, so they are measured on this card rather than read off a datasheet. The card does not reach its own spec sheet in either direction.
A roofline drawn from the datasheet puts the ridge point in the wrong place, and every verdict downstream moves with it. Measured here the ridge sits at 180.7 FLOP per byte. Anything to the left of it cannot be compute-bound however much arithmetic it does, which is what makes the compute-bound share in Figure 3 meaningful rather than assumed.
A CUDA graph replays the same work with the launches removed, so what it saves was never computation. Nearly 1,500 kernels go into one decode step.
Because the profiler inflates the very thing being measured. Instrumenting launches makes launches expensive, and at batch 1 it runs 1.83x slower than clean wall time. So the headline comes from the graph capture, where two real executions are compared and neither is being watched.
Every kernel gets filed against the measured ceilings: near the memory roof, near the compute roof, or far below both, which is latency-bound and not the same thing as memory-bound. Non-kernel time is what is left over.
Compute-bound is 0.0% at all 28 points. The compute ceiling is measured and never binds, which is the half of the folk claim that survives. The half that does not: calling everything else memory-bound hides that most of the step is below both ceilings, or outside the kernels.
These shares are profiled, so they carry the 1.83x inflation and the non-kernel slice reads larger than Figure 2 measures. Read the shape across batch and context, not the height.
Nine serving configurations over the same 200 prompts, greedy, one seed. Prefix caching and batching move bytes: up to 15 of 200 completions come back textually different. What is inside them almost never moves, and the score never moves at all.
Accuracy is 88.5% in all nine conditions. Not close to equal, equal. So byte-identical output is the wrong thing to gate a serving change on: it fails under exactly the settings you turn on for throughput, while the thing you actually care about is untouched.
The divergence is real rather than sampling noise. The same engine run five times over is byte-identical every time, so what moves the bytes is the batch a request lands in, not the decode.