Task selection

The task type is the first thing you choose and the hardest to change later. It decides the shape of your training rows, which metric your model is scored on, and which student and teacher models you’re allowed to use.

The value goes in base.task in your config.yaml:

base:
  task: classification

Choosing between them

If you need to	Choose	base.task
Assign text to one of a fixed set of categories	Classification	classification
Extract or generate an answer from text	Question answering	question-answering
Answer from a passage you supply at inference time	Open book QA (RAG)	question-answering-open-book
Answer from knowledge baked into the model	Closed book QA	question-answering-closed-book
Produce a tool call from a single request	Tool calling	tool-calling-closed-book
Produce tool calls across a conversation	Multi-turn tool calling	multi-turn-tool-calling-closed-book

Three of those values end in -closed-book. That means the knowledge lives in the model’s weights rather than in context you pass at inference time. Open book is the opposite: you supply the passage with every request, which is what a RAG pipeline does. Tool calling counts as closed book because the model learns your tool schemas during training instead of being handed them at inference.

Tool calling and multi-turn tool calling work with only some students and some teachers, so check Supported models before you commit. Switching task type later means rebuilding your data.

Every task type can be trained from production traces instead of a hand-built dataset. See Trace inputs.

Classification

classification

The model reads input text and assigns it one category from a fixed set you define. Pick it when you need a deterministic label rather than free text.

Your job_description.json needs a classes_description naming every class, and the label sets in train.jsonl, test.jsonl and classes_description have to match exactly. Scored on accuracy, plus per-class precision and recall.

Use cases. Intent detection for customer service, content moderation, sentiment analysis, topic categorization for a knowledge base, triaging support tickets by department.

Classification data preparation →

Question answering

question-answering

The model locates the relevant information and returns a targeted answer, rather than summarising or paraphrasing a whole document. The input carries everything needed to answer.

Scored by an LLM judge, so llm_as_a_judge_instructions in your job description does real work here.

Use cases. Pulling the termination clause out of a contract, the total due off an invoice, the decisions from meeting minutes, the root cause from an incident report.

Question answering data preparation →

Open book QA for RAG

question-answering-open-book

The model answers using a passage you supply alongside the question, staying grounded in that text rather than in general knowledge. This is the shape a retrieval-augmented pipeline needs: your retriever supplies chunks, the model answers strictly from them.

Rows carry a context field next to the messages, and unstructured.jsonl is required, so the teacher can draw on realistic passages when generating examples.

Pick it when you already have, or can retrieve, the passages the model answers from.

Use cases. Support answering from product documentation, legal document analysis, technical documentation assistants, FAQ automation over a knowledge base.

Open book QA data preparation →

Closed book QA

question-answering-closed-book

The model learns facts from your unstructured data during training, so questions get answered from its own weights with nothing passed in at inference.

unstructured.jsonl is required and does the heavy lifting: it’s the knowledge being moved into the model, and what it covers bounds what the model can answer.

Pick it when you have a lot of unstructured material and you’d rather not build and run a retrieval system.

Use cases. Answering product and service questions without a retriever, domain assistants over a fixed corpus.

Closed book QA data preparation →

Tool calling

tool-calling-closed-book

The model maps a request to a structured tool call with the right arguments, using only the request itself. It learns your tool schemas during training rather than being given them at inference.

Your job description carries a tools array in OpenAI function-calling format, and every call in your data validates against those schemas. Assistant rows carry empty content and a tool_calls array.

Pick it when you have a fixed set of tools and want schema-compliant dispatch.

Use cases. Mapping spoken commands to smart-home APIs, converting user intents to CRM operations, routing requests to microservices, parsing input into system commands.

Tool calling data preparation →

Multi-turn tool calling

multi-turn-tool-calling-closed-book

The same, inside a conversation. The model takes a history of alternating user and assistant messages and produces the next call, so users can issue follow-ups that build on earlier turns.

Each assistant turn produces exactly one call. Note that evaluation expands each conversation into one line per tool call, so your reported test-set size won’t match the number of rows you uploaded.

Pick it when commands depend on what came before.

Use cases. File system assistants, building database queries by refinement, running infrastructure commands conversationally, multi-step service requests.

Multi-turn tool calling data preparation →

Next

Prepare your data: from a seed dataset if you have labelled examples, or from traces if you have production logs.