2 · One question → next token · 22/38

Sampling

T flattens. top-p cuts the tail. Dice, not weights.

play
T flattens · top-p cuts the tail · sample is a dice roll, not a weight
id — · H — bits

The network ends at logits. T and top-p are serving policy.

The network ends at logits. Policy on those logits is serving.

chapter →Inference & decoding