Chat completion data preparation
The example on this page is an assistant for a weather and travel service: it chats with the user in free text, calls a tool when one helps, reads what the tool returned, and answers.
The shared rules for the input directory, row shapes and validation are in Overview. This page covers what’s specific to chat completion.
Chat completion covers every conversational model: a plain chat model with no tools, a model that only emits tool calls for your code to run, and an agent that reads the tool results and keeps going. Which one you get depends on the tools you declare and the conversations you train on.
Job description
Section titled “Job description”What you expect the model to do, in the words you’d use to prompt an LLM:
task_description. The task itself. There is no system-prompt field anywhere in the input: this text becomes the system prompt the model is trained and evaluated with, so write it with that care. Withbase.variable_system_prompton, each row carries its own system prompt instead (System prompts).tools. Optional. Every tool the model can call, in OpenAI function-calling format, with unique names. Leave it out for a pure chat model with no tools. If your data contains a tool call, the tool has to be declared here. Every call validates against these schemas, or against the row’s own list when it has one (Tools per row).llm_as_a_judge_instructions. Optional. Predictions are free text scored by an LLM judge, so this field does real work here.
{
"task_description": "You are a friendly assistant for a weather and travel service. Chat with the user, call a tool when it helps answer the request, and answer briefly and concretely from what the tools return. Stay on the topics of weather and travel planning.",
"tools": [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Look up the current weather for a location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "City to look up the weather for"
}
},
"required": ["location"]
}
}
},
{
"type": "function",
"function": {
"name": "search",
"description": "Search for information",
"parameters": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "Search query"
}
},
"required": ["query"]
}
}
}
]
}
Training data
Section titled “Training data”Each row is one whole conversation in a single messages array, with three roles:
userturns carry the request, incontent.assistantturns carrycontent,tool_calls, or both. A turn with neither fails validation.toolturns carry the result of a call:{"role": "tool", "tool_call_id": ..., "content": ...}. They may only follow an assistant turn that made tool calls. After one, the conversation continues with anothertoolresult, anassistantturn or the nextuserturn.
A conversation starts with a user message and ends with an assistant message. User and assistant
turns alternate, with tool results between them where a call returned something. No system
messages (the task description plays that part), unless
base.variable_system_prompt is on. A job without tools has no
calls, so its rows have only user and assistant turns. Production histories that break these
rules, such as an assistant greeting first or a system message mid-conversation, need
Flexible conversation structure.
Tool results are optional. Leave them out when your code runs the call and the model never sees
what came back; include them when the model reads the result before its next turn. The
tool_call_id on a result points at the id of the call it answers. A single call and a single
result are matched automatically; with parallel calls the ids must match. A turn can make two
calls and carry one result.
Written out, one conversation looks like this:
[
{"role": "user", "content": "What's the weather in Paris?"},
{"role": "assistant", "content": "Let me check that for you.", "tool_calls": [
{"id": "call_1", "type": "function", "function": {"name": "get_weather", "arguments": {"location": "Paris"}}}
]},
{"role": "tool", "tool_call_id": "call_1", "content": "18C, light rain"},
{"role": "assistant", "content": "It's 18C in Paris with light rain."}
]
In tool_calls, arguments is a JSON object, not a JSON-encoded string. The tool schemas in
job_description.json use the OpenAI function-calling format.
In train.jsonl that whole array goes on one line. Mix the shapes your production conversations
take: plain chat turns, a call then an answer from its result, chained calls where the second
depends on the first’s result, parallel calls, and calls whose result the model never sees.
There’s no minimum count (Dataset overview).
{"messages": [{"role": "user", "content": "What's the weather in Paris?"}, {"role": "assistant", "content": "Let me check that for you.", "tool_calls": [{"id": "call_1", "type": "function", "function": {"name": "get_weather", "arguments": {"location": "Paris"}}}]}, {"role": "tool", "tool_call_id": "call_1", "content": "18C, light rain"}, {"role": "assistant", "content": "It's 18C in Paris with light rain."}]}
{"messages": [{"role": "user", "content": "Look up the tallest building, then tell me the weather there."}, {"role": "assistant", "tool_calls": [{"id": "call_1", "type": "function", "function": {"name": "search", "arguments": {"query": "tallest building in the world"}}}]}, {"role": "tool", "tool_call_id": "call_1", "content": "Burj Khalifa, Dubai"}, {"role": "assistant", "tool_calls": [{"id": "call_2", "type": "function", "function": {"name": "get_weather", "arguments": {"location": "Dubai"}}}]}, {"role": "tool", "tool_call_id": "call_2", "content": "34C, sunny"}, {"role": "assistant", "content": "It's the Burj Khalifa in Dubai, where it's 34C and sunny."}]}
{"messages": [{"role": "user", "content": "Compare the weather in Rome and Madrid."}, {"role": "assistant", "content": "I'll check both.", "tool_calls": [{"id": "call_1", "type": "function", "function": {"name": "get_weather", "arguments": {"location": "Rome"}}}, {"id": "call_2", "type": "function", "function": {"name": "get_weather", "arguments": {"location": "Madrid"}}}]}, {"role": "tool", "tool_call_id": "call_1", "content": "22C, clear"}, {"role": "tool", "tool_call_id": "call_2", "content": "27C, sunny"}, {"role": "assistant", "content": "Madrid is warmer at 27C and sunny; Rome is 22C and clear."}]}
{"messages": [{"role": "user", "content": "Hi! Can you help me plan a trip?"}, {"role": "assistant", "content": "Of course. Where are you headed, and when?"}, {"role": "user", "content": "Lisbon, next week."}, {"role": "assistant", "content": "Let me check the weather there.", "tool_calls": [{"id": "call_1", "type": "function", "function": {"name": "get_weather", "arguments": {"location": "Lisbon"}}}]}]}
{"messages": [{"role": "user", "content": "Thanks, that's all I needed."}, {"role": "assistant", "content": "Happy to help. Ask me any time."}]}
The third row makes two calls in one turn. Parallel calls are valid data, but by default the
teacher generates at most one call per assistant turn; raise
synthgen.max_tool_calls_per_turn if your task needs more (see Config below).
A conversation can be split into one example per assistant turn, each predicted from everything before it, tool results included. That matches how the deployed model runs: your code executes the calls, appends the results, and calls the model again (Conversation expansion).
Test data
Section titled “Test data”Same format as train.jsonl, held out for evaluation. This is what every score you’ll see is
measured against: the teacher evaluation, the base student and the tuned student.
Cover the shapes that matter to you: turns that should chat, turns that should call each tool, a multi-step chain, answers written from a tool result, and follow-ups that depend on earlier turns.
{"messages": [{"role": "user", "content": "Is it warm enough for the beach in Barcelona?"}, {"role": "assistant", "tool_calls": [{"id": "call_1", "type": "function", "function": {"name": "get_weather", "arguments": {"location": "Barcelona"}}}]}, {"role": "tool", "tool_call_id": "call_1", "content": "29C, sunny"}, {"role": "assistant", "content": "Yes: 29C and sunny in Barcelona, a good beach day."}]}
{"messages": [{"role": "user", "content": "Where are the next Olympics, and what's the weather there?"}, {"role": "assistant", "tool_calls": [{"id": "call_1", "type": "function", "function": {"name": "search", "arguments": {"query": "next Olympics host city"}}}]}, {"role": "tool", "tool_call_id": "call_1", "content": "Brisbane, Australia (2032)"}, {"role": "assistant", "tool_calls": [{"id": "call_2", "type": "function", "function": {"name": "get_weather", "arguments": {"location": "Brisbane"}}}]}, {"role": "tool", "tool_call_id": "call_2", "content": "24C, partly cloudy"}, {"role": "assistant", "content": "They are in Brisbane, where it's 24C and partly cloudy."}]}
{"messages": [{"role": "user", "content": "What can you do?"}, {"role": "assistant", "content": "I can check the current weather for any city and search for travel information. What do you need?"}]}
With conversation expansion, the number of scored examples can exceed the number of rows you uploaded.
The main score is llm-as-a-judge-reference-free, which judges the text the model writes,
including the answers it writes after tool results. The tool-call metrics score the calls
against the reference (Metrics). Parallel calls are scored in order: a
turn with the right calls in the wrong order counts as a miss, so put the reference calls in the
order you want them made.
Tools per row
Section titled “Tools per row”When your application offers different tools in different requests, give each row the tools it
was offered: a tools list next to messages, in the same OpenAI function-calling format as the
job description.
{"messages": [{"role": "user", "content": "What's the weather in Oslo?"}, {"role": "assistant", "tool_calls": [{"id": "call_1", "type": "function", "function": {"name": "get_weather", "arguments": {"location": "Oslo"}}}]}], "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Look up the current weather for a location", "parameters": {"type": "object", "properties": {"location": {"type": "string"}}, "required": ["location"]}}}]}
{"messages": [{"role": "user", "content": "What's the weather in Oslo?"}, {"role": "assistant", "content": "I can't look up the weather right now, but I can help you plan the trip."}], "tools": []}
- A row without a
toolskey gets the job description’s tools. An empty list means the row offers no tools. - The row’s tool calls validate against its own list. Training, teacher evaluation, student
evaluation and the tool-call metrics use it too: the model is shown the row’s tools, and
tool_call_equivalencetakes argument defaults from them. - The job description still defines the tools. Every call has to name one of its tools and use
only argument names that tool declares, unless the tool sets
additionalProperties: true. A row’s list can offer fewer tools or a narrower definition of one, but a call to a tool the job description lacks fails validation. - Tool names are unique within a list.
- Synthetic rows are offered the job description’s tools.
Traces carry their tools the same way (Traces).
Flexible conversation structure
Section titled “Flexible conversation structure”Production agents send histories that the rules above reject: a system message inserted halfway,
for example a reminder or a changed instruction, an assistant greeting before the user says
anything, or two user or assistant turns in a row. To train on them as they are, turn on
base.flexible_conversation_structure (default false):
base:
task: chat-completion
variable_system_prompt: true
flexible_conversation_structure: true
With it on, rows may contain:
- system messages after the first turn, not only at the start;
- an assistant turn, such as a greeting, as the first turn;
- consecutive turns from the same role, such as two user messages in a row.
A conversation still needs at least one user message and still ends with an assistant message. A tool result still can’t open a conversation. It still follows an assistant turn that made calls, or another tool result, and no system message may come between them. Synthetic data generation and relabeling tell the teacher that these shapes are valid, so it can follow your examples instead of reshaping them.
The flag needs base.variable_system_prompt: true, since the leading system message is part of
the same history (System prompts). It’s rejected for students
whose chat template can’t render a system message that isn’t first: Qwen3.5, Qwen3.6, Qwen3.8 and
Gemma 3. Granite 4.2, Llama 3, LFM2.5 and FunctionGemma render all three shapes. Qwen3 and
Gemma 4 are accepted, but don’t train correctly on two assistant turns in a row. With
conversation expansion, only assistant turns
after the first user turn become examples, so a leading greeting stays history.
Config
Section titled “Config”The task type, the two models and the judge:
base:
task: chat-completion
student_model_name: Qwen3.5-4B
teacher_model_name: zai.glm-5.3-low-thinking
evaluation:
llm_as_a_judge_model_name: zai.glm-5.3-low-thinking
If the job description declares tools, both models have to be tool-calling capable (Supported models). With no tools declared, any supported model works.
One parameter is specific to this task:
synthgen:
max_tool_calls_per_turn: unlimited
It caps how many tool calls a generated assistant turn may contain. Unset, the limit is 1 when
tools are declared and 0 when they aren’t. Set an integer or unlimited to allow parallel
calls, or 0 to forbid calls outright. Declaring no tools while setting it above zero fails
validation, as does declaring tools while setting it to 0.
Every other field has a default. See Config file for the full table and Supported models for the values you can use.
Synthetic data with tools
Section titled “Synthetic data with tools”Synthetic data generation does not target tools one at a time: every generation call sees all the declared tools, and the teacher decides which tool each example uses, so some tools can end up rare or missing. When the job description declares tools, add a mutator with one value per tool, so every generation call is asked for a specific one:
synthgen:
mutators:
- name: tool
values:
- "the conversation calls get_weather"
- "the conversation calls search"
- "the assistant answers in text with no tool call"
The last value covers turns that should answer without a tool; leave it out if every turn should
call one. Set target_distribution: match_seed to follow the tool mix of your training rows
instead of an even split, or give one weight per value. Count the tools in the output of a
smoke run before the full run
(Synthetic data generation).
Put the files in one directory, here input-dir, and create the dataset from it. This
validates your files at the same time:
distil dataset create --data input-dir
The command prints Upload successful. Dataset ID: <dataset-id>. Run
teacher evaluation on that ID next.