The Six Task Types for Training a Small Language Model
A task type is the training recipe you declare in config.yaml, and there are six: classification, question answering, open-book QA, closed-book QA, tool calling, and multi-turn tool calling. It sets the shape of your training files, the metric you are scored on, and which student models you may use.
What are the six task types?
They are the six recipes listed in the task selection guide, each with an exact string you put in base.task.
| Task type | base.task value |
What the model produces |
|---|---|---|
| Classification | classification |
One label from a fixed set |
| Question answering | question-answering |
A targeted answer extracted or generated from the input |
| Open-book QA (RAG) | question-answering-open-book |
An answer grounded in a passage you pass in |
| Closed-book QA | question-answering-closed-book |
An answer from knowledge learned during training |
| Tool calling | tool-calling-closed-book |
One structured function call |
| Multi-turn tool calling | multi-turn-tool-calling-closed-book |
The next function call, given conversation history |
There is no seventh option, so anything that is not a label or a function call maps to one of the question-answering variants. Summarisation, rewriting, and structured extraction all land on question-answering, which the docs describe as the fit for transforming text “by addressing implicit questions about its content” — the task type describes the shape of the input-output pair, not the subject matter.
How does the task type change your data files?
It changes which files you must supply and what each JSONL line contains. Every task uses the same four-file directory (job_description.json, train.jsonl, test.jsonl, config.yaml), but the contents differ enough that a file prepared for one task will not train another.
| Task type | Extra field in job_description.json |
Shape of a train.jsonl line |
|---|---|---|
| Classification | classes_description map |
messages: user text → assistant label |
| Question answering | input_description |
messages: user question → assistant answer |
| Open-book QA | none required | messages plus a sibling context field |
| Closed-book QA | none required | messages; unstructured.jsonl carries the knowledge |
| Tool calling | tools (OpenAI function schemas) |
assistant turn with tool_calls, no content |
| Multi-turn tool calling | tools |
whole conversation in one messages array |
Two details trip people up. Closed-book QA is the one task where unstructured.jsonl is not optional — it is the only channel through which facts enter the model. And tool calls are emitted in the HuggingFace format, where arguments is a real JSON object, while the schemas in job_description.json follow OpenAI’s function-calling format, where they are not. Getting these backwards is the most common data-prep failure on tool-calling tasks.
Why does the task type restrict which models you can use?
Because tool calling requires structured-output behaviour that not every base model has. The supported models catalog limits tool-calling-closed-book and multi-turn-tool-calling-closed-book students to the Qwen3, Qwen3.5, Llama 3, LFM2/LFM2.5, FunctionGemma, and Gemma 4 families, and limits teachers to those with a tick in the tool-calling column.
That has a practical consequence: pick the task before the model. Choosing Gemma 3 1B for a classifier and later deciding it should also route function calls means starting over with a different student — the restriction is enforced at validation time, not degraded gracefully.
How is each task type scored?
By a metric matched to the output shape, which is why the task type also fixes what “accuracy” means for you. Text-producing tasks (classification, all three QA variants) are scored with exact match, ROUGE-L, and an LLM-as-a-judge score; distil labs recommends the judge because it credits a semantically correct answer with different wording. The research behind that approach is surveyed in this arXiv paper.
Tool-calling tasks are scored on tool call equivalence instead, with a staged_tool_call variant that awards quarter-credit for valid JSON, the right function name, the right parameter keys, and an exact match. That staged breakdown is genuinely useful during development: a score of 0.5 means the model picked the right function and filled the arguments wrong, which is a different bug from a score of 0.25.
When do two task types both look right?
More often than you would expect, and the tie-breaks are worth knowing.
- Classification versus question answering. If the output is a closed set of labels, use classification. Framing it as generation throws away accuracy and makes the result harder to measure.
- Open-book versus closed-book QA. Open-book if you can retrieve the passage at inference time and need to cite it; closed-book if the knowledge is stable and you want one self-contained artefact. The full trade-off is in open-book vs closed-book QA.
- Tool calling versus multi-turn. If users say “actually, make that Tuesday”, you need multi-turn. Single-turn training will not get you there.
When you are still unsure, which task type should you pick walks the decision row by row.
Related terms
A student model is the small model being fine-tuned; a teacher model generates and validates the synthetic training data. Both are explained in what is a student model and what is a teacher model. Teacher evaluation is the pre-training check that a large model can solve your task at all. Synthetic data generation is what expands your few dozen seed examples into thousands — see generating synthetic training data.