Tools
LLM sampling
A distribution of candidate tokens, and the four settings that reshape it before the draw. Move the temperature and watch the gap between candidates open or close, cut with top-k or top-p, then draw a hundred samples to compare observed frequency against theoretical probability. Everything is computed in the browser, no model is called.
The model produces one raw score (a logit) per token in the vocabulary. The settings below turn those scores into probabilities, then cut part of them away. The draw happens in whatever is left.
Divides every logit before the softmax. Below 1 the gap between candidates widens and the best one dominates; above 1 it flattens and long shots get a real chance. At 0 the model always picks the top token.
Keeps only the k most likely candidates and throws the rest away, whatever their probability. A blunt cut: the same k is too tight on an ambiguous step and too loose on an obvious one.
Keeps the smallest group of candidates whose probabilities add up to p. Adapts to the step: when the model is sure, it keeps one or two; when it hesitates, it keeps many. This is the one most people should tune.
Drops anything below this fraction of the best candidate. At 0.1, a token needs at least a tenth of the leader's probability to survive. A newer alternative to top-p that behaves better at high temperature.
Candidates kept
12 / 12
Most likely
Paris
96.3 %
Entropy
0.32 bits
near deterministic
Draws
0
none yet
- ␣Paris96.3%
- ␣the1.6%
- ␣located0.8%
- ␣a0.5%
- ␣home0.3%
- ␣known0.2%
- ␣one0.1%
- ␣situated0.1%
- ␣Lyon0.0%
- ␣Marseille0.0%
- ␣Berlin0.0%
- ␣potato0.0%
Probabilities are renormalised over the survivors, which is what the sampler actually draws from. Push the temperature past 1.5 and watch "potato" become a real possibility.