What Is Temperature in LLMs?
Temperature is a divisor applied to the logits before softmax. Below 1 it widens the gaps between scores, concentrating probability on the top token; above 1 it flattens them, giving unlikely tokens a real chance. At 0 the model becomes deterministic, always taking the argmax.
What does it do to the distribution?
It rescales every score by the same divisor, which changes how much the leading token dominates.
| Setting | Effect on the distribution | Typical use |
|---|---|---|
| 0 | Argmax only, no sampling | Reproducible extraction, classification |
| Below 1 | Sharpened toward the leading token | Structured output, tool calls |
| 1 | The model’s own distribution, unchanged | Neutral baseline |
| Above 1 | Flattened; tail tokens surface | Rarely useful; degrades coherence |
Temperature is only one of several decoding controls, and not the one that fixes repetition. Holtzman et al. showed in The Curious Case of Neural Text Degeneration that likelihood-maximising decoding produces bland, repetitive text, and proposed nucleus sampling as the remedy. The generation strategies guide covers how the two combine.
What should a teacher run at?
Between 0.5 and 0.7 if it’s a reasoning model. The supported models catalogue states that reasoning teacher families (GPT OSS, DeepSeek, GLM, Kimi and MiniMax) require synthgen.teacher_temperature inside that range, and configurations outside it raise a validation error.
Why can a valid-looking config still fail?
Because two docs pages bound the same field differently. The parameter table in the configuration reference lists teacher_temperature as a float that must sit between 0.0 and 1.0. Read that table alone, set 0.9, pair it with a reasoning teacher, and the job is rejected, because the narrower family-specific rule wins.
Since most current teachers are reasoning models, treat 0.5–0.7 as the working range whenever you’re choosing a teacher or generating synthetic data with one.