Task selection

The task type is the first thing you choose and the hardest to change later. It decides the shape of your training rows, which metric your model is scored on, and which student and teacher models you can use. Every task type can be trained from production traces as well as from a labeled dataset.

The value goes in base.task in your config.yaml:

base:
  task: classification

Choosing between them

The output is	Choose	base.task
One label from a fixed set	Classification	classification
Free text for a single input: answers, extractions, transformations, JSON documents	Question answering	question-answering
A conversational turn: free text, tool calls, or both; the model never sees tool results	Chat completion	chat-completion
An agentic loop: tool calls whose results feed back, ending in an answer	Agentic chat completion	chat-completion-agentic

question-answering covers any text-in, text-out problem, without the label checks that classification gets. When the input carries a document or retrieved text, put that text inside the user message. Data with tool results in it is chat-completion-agentic.

Chat completion with tools declared works with only some students and teachers, so check Supported models before you choose it.

Classification

classification

The model reads input text and assigns it one category from a fixed set you define. Pick it when you need a label rather than free text.

Your job_description.json needs a classes_description naming every class, and the label sets in train.jsonl, test.jsonl and classes_description have to match exactly. Scored on accuracy, with per-class precision and recall.

Use cases. Intent detection for customer service, content moderation, sentiment analysis, topic categorization, triaging support tickets by department.

Classification data preparation →

Question answering

question-answering

The model takes one input and returns one free-text answer. The input carries everything needed to answer, including any document or retrieved passage.

Scored by an LLM judge on llm-as-a-judge-reference-free, so llm_as_a_judge_instructions in your job description decides what counts as a correct answer.

Use cases. Pulling the termination clause out of a contract, the total due off an invoice, the decisions from meeting minutes, answering from a retrieved passage.

Question answering data preparation →

Chat completion

chat-completion

A multi-turn conversation where assistant turns are free text, tool calls, or both. The model chats with the user and calls a tool when one helps. Tools are optional: leave them out of the job description and you get a conversational model without tools.

The model never sees tool results in this task: an assistant turn is always followed by the next user turn. If your conversations feed tool outputs back to the model, use agentic chat completion.

Scored by an LLM judge on llm-as-a-judge-reference-free, with tool-call metrics alongside it (Metrics).

Use cases. Customer support chat that occasionally looks something up, assistants that mix conversation with actions, replacing a general chat model that has a handful of functions.

Chat completion data preparation →

Agentic chat completion

chat-completion-agentic

Chat completion plus tool results: the model calls tools, reads the results fed back as tool messages, and continues with another call or a final text answer. Parallel calls in one turn are supported.

tools is required in the job description, and your data carries the full loop: assistant tool calls, tool results, and the closing answer. Scored like chat completion.

Use cases. Research assistants that search and then summarize, support agents that look up an order before answering, any agent whose answer depends on what a tool returned.

Agentic chat completion data preparation →

Next

Collect traces with an endpoint, start from a trace file, or prepare a labeled dataset.