Overview

A seed dataset is a few dozen labelled examples plus a description of the task. The teacher grows it into a full training set later, so this is the smallest amount of data you have to write by hand.

Read this page before any task-specific one. It covers what every seed dataset needs, whatever the task type.

If you’re starting from production traces instead, you don’t need this page. Trace processing produces a valid seed dataset for you, so go to Trace inputs.

The input directory

input-dir/
├── train.jsonl             # seed training examples
├── test.jsonl              # held-out evaluation set
├── job_description.json    # what the task is, in words
├── config.yaml             # task type and models
└── unstructured.jsonl      # required for open-book and closed-book QA

This directory is the job input. There’s no separate compiled artifact, and edits to the files take effect directly. All data files are JSONL.

Create the seed dataset by pointing at the directory:

distil seed-dataset create --data ./input-dir
# Seed dataset created with ID: <seed-dataset-id>

Training and test data

Train and test rows are chat-format conversations.

Task	Row shape
classification, question-answering, question-answering-closed-book	{"messages": [<user>, <assistant>]}
question-answering-open-book	{"messages": [<user>, <assistant>], "context": "..."}
tool-calling-closed-book	{"messages": [<user>, <assistant with tool_calls>]}
multi-turn-tool-calling-closed-book	{"messages": [<user>, ..., <assistant with tool_calls>]}

<user> and <assistant> are {"role": ..., "content": "<non-empty>"}, and the final assistant message carries the answer or label as its content.

Tool-call assistant messages are the exception. Their content is empty, and the call goes in tool_calls:

{"messages": [
  {"role": "user", "content": "What's the weather in Berlin?"},
  {"role": "assistant", "content": "", "tool_calls": [
    {"type": "function", "function": {"name": "get_weather", "arguments": {"city": "Berlin"}}}
  ]}
]}

The key is arguments and it’s a JSON object, not a string.

unstructured.jsonl rows are {"context": "..."}.

There’s no hard minimum row count. Aim for 20+ diverse training examples and a test set that covers your production distribution. Full worked examples are on the task-specific pages linked at the bottom.

Job description

What your task is, in words. The teacher reads it when it’s evaluated, and again when it generates your training data.

For a seed dataset you usually want two fields:

{
  "task_description": "...",
  "llm_as_a_judge_instructions": "..."
}

• task_description describes the task itself: what the model should do, what a correct answer looks like, how the output should be formatted. Derive it from the system prompt of the system you’re replacing.
• llm_as_a_judge_instructions tells the judge what to accept when it scores an answer. Optional, and not valid for classification, which is judged on label accuracy instead.

Classification tasks also need classes_description, and tool-calling tasks need tools. Both are covered on the task-specific pages.

You don’t need trace_processing_instructions here, since that field is only read when starting from traces. The full field reference is Job description.

Config

The task type and the two models:

base:
  task: classification
  student_model_name: Qwen3-0.6B
  teacher_model_name: openai.gpt-oss-120b

Everything else has a default. See Config file for the full table.

Validation rules

These are enforced when the job parses your directory, so check them before submitting.

• Train and test are non-empty, and every message content is non-empty. The exception is tool-call assistant messages, whose content is empty by design.
• context is present when the task requires it.
• Per row, total length is at most synthgen.validation_max_total_length.
• Train and test share no identical rows. Exact duplicates fail.
• unstructured.jsonl is required for question-answering-open-book and question-answering-closed-book, and optional elsewhere. When present it holds non-empty string context rows, at least synthgen.num_unlabelled_exemplars_per_generation of them.
• For classification, the label sets in train.jsonl, test.jsonl and classes_description are identical, and each class has at least the configured number of exemplars (default 2).
• For tool calling, every call validates against the tools schemas. Multi-turn conversations start with a user message, end with an assistant tool call, and have a valid role sequence.

Creating the seed dataset is how you check all of this. distil seed-dataset create parses the directory exactly as the jobs will, with full schema validation and no model calls, and rejects an invalid bundle with the reason before any job is created. The route is metered, but only a successful create spends a credit, so validating a broken bundle repeatedly costs nothing.

Read the reason rather than just the failure. The three shapes look different and all name the cause:

# a file missing from the directory, caught locally, nothing uploads
Required file not found: test.jsonl in input-dir

# a task the platform does not have
Only the following tasks are supported: ['classification', 'question-answering-open-book', …]

# a row in the wrong shape, named down to the row index
1 validation error for JobInputParser
job_input.1.data.question-answering.train_dataset.0.messages
  Field required [type=missing, input_value={'wrong': 'shape'}, input_type=dict]

Each exits 1 and creates nothing. Correct the directory and run the command again.

Next

Pick your task’s page for the exact row format and a worked example:

• Classification
• Question answering
• Closed book QA
• Open book QA for RAG
• Tool calling
• Multi-turn tool calling

Then run teacher evaluation.