Code Walk F · the modelreference · performance model

The roofline
the one diagram the company rests on

Every other lab is a special case of this chart. A GPU has two ceilings — how fast it can compute, and how fast it can move memory — and which one you hit depends on a single number: arithmetic intensity, the FLOPs you do per byte you fetch. Plot achievable throughput against it and you get the roofline: a sloped memory-bound wall rising to a flat compute ceiling. The whole plasmient thesis is one sentence about this picture — nvidia-smi tells you the GPU is busy; the roofline tells you whether that busy is worth anything. Drag the probe.

the two ceilingspeak FLOP/s · memory bandwidth
the one numberarithmetic intensity — FLOPs per byte
the ridgewhere memory-bound meets compute-bound
the thesismost real ops live left of it

00 Two ceilings, one number

A GPU can do two things fast: arithmetic (multiply-add, in the tensor cores) and moving bytes (streaming weights and activations to and from HBM). Every kernel is limited by one or the other. Which one is set by a single ratio — arithmetic intensity, the number of FLOPs performed per byte fetched from memory. Low intensity means you fetch a lot and compute little: you’rememory-bound, waiting on HBM while the tensor cores idle. High intensity means you reuse each byte many times: you’re compute-bound, and the silicon runs flat out.

Plot the best throughput you could possibly achieve against intensity and you get the roofline: a diagonal wall (bandwidth × intensity) rising until it hits a flat ceiling (peak FLOP/s). They meet at the ridge. Left of the ridge, you’re memory-bound and can never reach peak, no matter how good your code — the bandwidth simply can’t feed the cores. Right of it, you’re compute-bound and peak is on the table. Pick a GPU, then drag the probe or click an op:

The roofline for three GPUs. The diagonal is the memory ceiling (bandwidth × intensity); the flat top is the compute ceiling (peak FLOP/s); the dashed line is the ridge where they meet. Each dot is a real op at its arithmetic intensity — most of them sit far left, achieving a few percent of peak. Drag the probe across the x-axis and watch the achievable fraction of peak collapse as you move into the shaded memory-bound region.

GPUDrag the probe, or click an op. Left of the ridge = memory-bound = a fraction of peak.

Notice where the ops land. Elementwise adds, RMSNorm, and single-token decode attention sit deep in the memory-bound region — a few percent of peak is the ceiling, the best any kernel could do. Only the big training GEMMs and prefill attention live right of the ridge, where the datasheet number is actually reachable. Switch from H100 to RTX 4090 and the ridge slides left: a different card makes the same op a different problem.

01 Why the ridge is so far right

Here’s the part that surprises people. The H100’s ridge sits at roughly peak ÷ bandwidth ≈ 989 ÷ 3.35 ≈ 295 FLOPs per byte. That means a kernel must do nearly 300 floating-point operations for every byte it reads just to reach the compute ceiling. Most ops don’t come close. Reading one bf16 number (2 bytes) and adding it to another does 1 FLOP for 4 bytes moved — intensity 0.25. It is memory- bound by a factor of a thousand, and there is nothing you can do in the kernel to change that; the work itself has no reuse.

the uncomfortable arithmeticAs GPUs get faster, peak FLOP/s has grown far quicker than memory bandwidth. So the ridge keeps movingright — the intensity you need to be compute-bound keeps rising. Newer silicon makes moreof the workload memory-bound, not less. The gap the company measures widens with every generation.

The roofline above plots the outcome. Here is the mechanism that produces it. Bytes leave HBM on a conveyor that runs at one fixed speed — the bandwidth ceiling, always saturated. Arithmetic intensity is the only knob that decides what happens next: it sets how many FLOPs each arriving byte is worth. Turn it down and the tensor cores starve, blinking idle while the belt runs full; turn it up and the cores light solid while the memory lane finally has slack. Pick an op and watch the coupling:

The same H100 (peak 989 TFLOP/s, 3.35 TB/s HBM, ridge ≈ 295 F/B) seen as a machine. Bytes stream in on the memory lane at a fixed rate; each is worth × intensity FLOPs to the tensor cores below. At low intensity the cores sit almost entirely idle — waiting on bytes — so the chip achieves a few percent of peak while every byte-mover is pinned at 100%. That idle-but-busy gap is exactly what nvidia-smi cannot see and what the roofline names. Only when intensity crosses the ridge do the cores saturate — and the memory lane starts showing slack.

op · intensityPick an op. Low intensity starves the cores; high intensity leaves the memory lane with slack.

This is why FlashAttention matters and why it’s in this stack twice: it doesn’t reduce the math — it raises the intensity of attention by never writing the big intermediate matrix to HBM, moving the op rightward on this exact chart. Every serious kernel optimization is, in the end, a move on the roofline: either toward the ceiling you’re under, or across the ridge to a better one.

02 What nvidia-smi does — and doesn’t — see

The roofline is a claim about achievable throughput. The utilization number on your dashboard is a claim about something else entirely: the fraction of recent time the GPU had at least one kernel resident. A memory-bound kernel that stalls on HBM the entire time it runs still counts as “busy” — the SM is occupied, waiting. So nvidia-smi reads 100% at both ends of this chart, for the elementwise add achieving 0.3% of peak and the big GEMM achieving 90%.

  op              intensity   roofline says      nvidia-smi says
  ─────────────   ─────────   ────────────────   ───────────────
  elementwise +      0.13     ~0.3% of peak       100% busy
  RMSNorm            0.4      ~1%   of peak        100% busy
  decode attention   1.2      ~4%   of peak        100% busy
  FlashAttention     16       ~35%  of peak        100% busy
  prefill / GEMM     160      ~90%  of peak        100% busy
                              └─ 300× spread ─┘    └ one number ┘

That column on the right is the entire problem. One number, flat across a 300× spread in useful work. The roofline is how you recover the missing axis: place the op by its intensity, read off the ceiling it’s under, and you know whether “busy” meant anything.

03 Why an observability company rests on this chart

the plasmient thread

Every other lab here is a point on this plot. The CUDA matmul written twelve ways is one op climbing toward the compute ceiling. FlashAttentionis an op stepping right across the ridge. Serving is the fight to keep the memory-bound decode phase from wasting the compute ceiling it can’t use. Grouped-query attention shrinks the bytes so the same op moves rightward. They are all moves on this one diagram.

And the one number the industry watches — utilization — is the one axis this diagram doesn’thave. That’s the wedge. Plasmient instruments the missing axis: it reads each kernel’s real arithmetic intensity and achieved FLOP/s, places it on the roofline for the actual card it ran on, and reports the fraction of peak next to the utilization everyone already trusts. The gap between them is the product. This is the diagram the company rests on — the rest of theFrontier Stack is just where each op happens to land.

↗ Full references — go to the source

Start with Williams’ original roofline paper and Horace He’s ML translation of it. Then Nsight Compute is where you read intensity off a real kernel, and nvbandwidth is how you measure your own ceiling instead of trusting a datasheet.