How to Choose a Teacher Model for Distillation

Choose the teacher that scores highest on your held-out set, not the one with the most parameters. Start from zai.glm-5 — it is what recent distil labs work uses — and run a teacher evaluation on two or three candidates before training. Tool-calling tasks need a tool-calling-capable teacher.

What criteria actually matter?

Four, in this order: task compatibility, measured accuracy on your data, reasoning style, and cost of the generation run.

Criterion	What to check	Which teacher
No strong signal either way	—	zai.glm-5
Task type	Does the catalog mark it tool-calling capable?	zai.glm-5, deepseek.v3.2, Qwen3-235B-A22B-Instruct-2507, moonshotai.kimi-k2.5, minimax.minimax-m2-thinking, openai.gpt-oss-120b
Accuracy on your data	Teacher evaluation score on your held-out set	Whichever wins; nothing else settles it
Multi-step reasoning	Does the task need chain-of-thought before the answer?	A Thinking variant, e.g. moonshotai.kimi-k2-thinking or openai.gpt-oss-120b-thinking
Vision inputs	Are inputs images rather than text?	Qwen2.5-VL-72B-Instruct

Two defaults exist and they disagree, which is worth knowing before you copy one. base.teacher_model_name defaults to openai.gpt-oss-120b, which the docs call “a strong choice for all task types.” trace_processing.teacher_model_name defaults to zai.glm-5. Recent work has converged on the latter for both slots.

The evidence is measured rather than editorial. In our traces-vs-synthetic benchmark, five teachers were scored on the same held-out multi-turn tool-calling set: GLM-5 led at 0.835 LLM-as-a-judge, ahead of Qwen3-235B at 0.768, GPT-OSS-120B at 0.765, MiniMax-M2 at 0.762 and DeepSeek-3.2 at 0.744. GLM-5 is also the teacher in the Claude skill trace walkthrough and the deferral work.

That nine-point spread is on one task, so treat it as a shortlist and not a ranking. The supported models catalog is the authoritative list — a teacher_model_name outside it will not train.

A working starting config, with the temperature already inside the range reasoning teachers require:

base:
  task: multi-turn-tool-calling-closed-book
  student_model_name: Qwen3-1.7B
  teacher_model_name: zai.glm-5

synthgen:
  teacher_temperature: 0.6

Does your task need a tool-calling teacher?

If you selected tool-calling-closed-book or multi-turn-tool-calling-closed-book, yes, and the constraint is hard rather than advisory.

The catalog carries a Tool calling column, and several strong teachers are marked ✗. DeepSeek R1 and DeepSeek V3.1 do not qualify, but V3.2 does. Qwen3-480B-A35B-Coder does not, despite being the largest model in the list, while Qwen3-235B-A22B-Instruct-2507 does. Picking on size alone will pick you a teacher that cannot emit the function calls you need.

If you are unsure which task type applies, task selection walks the six options.

How do you verify a teacher before training?

Run teacher evaluation, which asks the candidate to solve your task on the held-out set and reports the same metrics your student will later be scored on.

distil model run-teacher-evaluation <model-id>
distil model teacher-evaluation <model-id>

Two things come out of it. The first is a go/no-go signal: docs state that “if the teacher model can solve the task, the student model will be able to learn it effectively,” and if it cannot, you refine your inputs before spending a training run. The second is a target — the teacher’s score is the first approximation of what the student will reach.

Download the per-example predictions when the aggregate score is ambiguous:

distil model download-teacher-evaluation-predictions <model-id>

Reading twenty wrong predictions usually tells you whether the teacher is weak or your task description is. Full detail is on the teacher evaluation page.

What are the common mistakes?

Six recur often enough to be worth naming.

Assuming bigger is better. In our platform benchmark, the teacher scored 0.50 on the pizza tool-calling task while the distilled 3B student reached 0.70. A large teacher that is weak on your specific schema is still weak.

Skipping teacher evaluation. It is the cheapest step in the pipeline and the only one that can save you a full training run. Skipping it is how a badly specified task becomes an expensive badly specified task.

Setting only one of the two teacher slots. If you train from production traces, trace_processing.teacher_model_name picks the best relabel from the committee and base.teacher_model_name generates the synthetic data. They are separate keys with separate defaults, and leaving the trace-processing one unset is a real and easily missed bug — the Claude skill walkthrough hit exactly this and set both to zai.glm-5.

Setting an out-of-range temperature. Reasoning teachers — GPT OSS, DeepSeek, GLM, Kimi and MiniMax — require synthgen.teacher_temperature between 0.5 and 0.7. The parameter itself accepts 0.0 to 1.0 in the config reference, so a value that looks legal there will still fail validation with one of these teachers. Since that list covers every current recommendation, treat 0.5–0.7 as the real range.

Choosing a teacher for a benchmark it wins. Public leaderboards such as the Berkeley Function Calling Leaderboard are useful for shortlisting, but they measure generic function catalogs, not yours. Your held-out set is the only ranking that counts.

Treating the teacher score as a ceiling. It is a first approximation, not a limit. Students routinely pass it — can a small model beat its teacher has the figures.

When should you change teacher mid-project?

When teacher evaluation comes back low and you have already tightened the task description and cleaned the seed examples.

Order matters here. Docs list revising the task description, improving example quality, checking for dataset inconsistencies, and confirming the task is solvable before anything else. A different teacher will not fix an ambiguous label set. If the task genuinely is well specified and the teacher still fails, switch — and if every candidate fails, read when does distillation fail before continuing.

For background on what the teacher does in the pipeline at all, see what is a teacher model.