Load K2 / V3-class

INT4, TP=8. Smoke one completion.

play

VRAM follows total params · FLOPs follow active · 80 GB per H100

weights —HBM —active compute ——

K2 INT4 on 8×H100. Flip to 1× or FP16 — OOM. Active 32B is not VRAM.

1T resident on 8×H100 INT4. Active ~32B. One GPU OOM.