Building the train set
The train set is what the student learns from. It is built on the test-set dataset with the teacher that scored best in teacher evaluation, real rows first and synthetic rows second:
- Relabel traces into the train split. Production inputs with answers rewritten by the teacher. Generation imitates these, so they set the format, the style and the hard cases.
- Generate synthetic train rows. The bulk of the set, generated from the rows in the split, the job description, the mutators and the traces left as context. The default target is 10,000.
Both are the expand operations described on
Relabeling and synthetic data. Both leave test.jsonl unchanged,
so the model trained on the result is scored on the locked test set. A
vision model has the first only: synthetic data generation is rejected for visual tasks.
How many rows
Section titled “How many rows”Two counts:
trace_processing.num_train_relabelledis how many of the remaining traces become real train rows. Default 200; more when the pool is large. Generation wantsT + min(T, 1000)traces as context for a target of T, takes what is left otherwise, and generates without context belowmin(T / 4, 10)(The expand operations). With a few hundred traces left, relabeling most of them and generating with little context is the better trade: a relabeled row is worth more than a context trace.synthgen.train_generation_targetis the synthetic rows to add. 10,000 unless you want a different size.
Leave enough of your balance to train on the data afterwards. --dry-run prices the relabeling
and generation runs, and distil billing balance shows what you have
(Billing).
Set the teacher
Section titled “Set the teacher”The test-set dataset’s config is fully expanded, which means it names the relabeling teacher on
its own in trace_processing.teacher_model_name. So the teacher that passed evaluation goes into
both fields of the override. Download the config and set them:
distil dataset download-metadata -d train-set <test-dataset-id>
# in train-set/config.yaml:
# base.teacher_model_name: <the teacher that passed>
# trace_processing.teacher_model_name: <the same>
# trace_processing.num_train_relabelled: 200
# synthgen.train_generation_target: 10000
An override replaces the file whole, so edit the downloaded copy rather than writing a new one (Overrides).
Relabel traces into the train split
Section titled “Relabel traces into the train split”Run the expand on the test-set dataset. Skip it when no traces are left, or when you uploaded a train set and have no traces:
distil dataset relabel-traces-train --config train-set/config.yaml <test-dataset-id>
distil dataset status <train-dataset-id>
Generate synthetic train rows
Section titled “Generate synthetic train rows”Run generation on the dataset relabeling produced (or the test-set dataset when you skipped
relabeling), smoke first. A smoke generates 128 rows as a side branch so you can check the data
before generating all of it. Copy train-set/config.yaml to train-set/smoke.yaml, set
synthgen.train_generation_target: 128 in the copy, and submit it:
distil dataset generate-synthetic-data-train --config train-set/smoke.yaml <train-dataset-id>
Read the smoke’s rows as described in
Checking what came back, change one
setting at a time in train-set/smoke.yaml until a smoke passes, then carry those settings over
to train-set/config.yaml and run the full job on the same parent:
distil dataset generate-synthetic-data-train --config train-set/config.yaml <train-dataset-id>
Check the train set
Section titled “Check the train set”Download the final dataset and read the whole train split the same way: form against the job description, distribution against the relabeled rows and the traces, and the slices you asked for. Check the size against the plan. If it passes, the dataset goes to model training. If it doesn’t, don’t train on it: bad data shows up in training only as a score that stops improving, after a full training run. Fix the operation that produced the bad rows and run it again on its parent.
Topping up
Section titled “Topping up”A later iteration that needs more rows doesn’t rebuild. Run generation on the train split of the
dataset your best model was trained on, with train_generation_target set to the rows to add:
distil dataset download-metadata -d top-up <best-dataset-id>
# in top-up/config.yaml: synthgen.train_generation_target: 2000, plus the new mutators
distil dataset generate-synthetic-data-train --config top-up/config.yaml <best-dataset-id>
The job generates from every row already in the split, relabeled and synthetic alike, and deduplicates against them, so the output is the old rows plus the new ones. Any traces left serve as context; after a full run there are usually none, and the top-up generates without context.
One exception: a train split that generation split into per-turn rows (clean_training_targets
or enable_thinking on a multi-turn task) can’t be the examples of a later run. Top up on the
dataset before that run instead, with the full target.
What can be changed
Section titled “What can be changed”- The teacher. A different teacher is checked by teacher evaluation first; then both operations run again on the test-set dataset.
- The relabeled rows. Who rewrites them, with what guidance, how many (Relabeling). Generation imitates them, so better relabeling raises the whole set.
- The synthetic rows. Instructions, mutators, the
synthgenconfig, or a top-up (Synthetic data generation). - Hand-corrected rows. Download the dataset, fix
train.jsonl, upload the directory as a new dataset withdistil dataset create --data.
Cheapest first: a top-up or a hand correction, then generation again, then relabeling again (and generation after it), then a new teacher (teacher evaluation, then both).