Building the train set

The train set is what the student learns from. It is built on the test-set dataset with the teacher that scored best in teacher evaluation, real rows first and synthetic rows second:

1. Relabel traces into the train split. Production inputs with answers rewritten by the teacher. Generation imitates these, so they set the format, the style and the hard cases.
2. Generate synthetic train rows. The bulk of the set, generated from the rows in the split, the job description, the mutators and the traces left as context. The default target is 10,000.

Both are the expand operations described on Relabeling and synthetic data. Both leave test.jsonl unchanged, so the model trained on the result is scored on the locked test set.

How many rows

Two counts:

• trace_processing.num_train_relabelled is how many of the remaining traces become real train rows. Default 200; more when the pool is large. Generation wants T + min(T, 1000) traces as context for a target of T, takes what is left otherwise, and generates without context below min(T / 4, 10) (The expand operations). With a few hundred traces left, relabeling most of them and generating with little context is the better trade: a relabeled row is worth more than a context trace.
• synthgen.train_generation_target is the synthetic rows to add. 10,000 unless you want a different size.

Generation is wasted if no training credit is left after it, so read distil credits-balance against the whole plan: one datasets_from_datasets_relabel_traces_post, one datasets_from_datasets_generate_synthetic_data_post, the smoke credits, and the slms_from_datasets_post credit training will need.

Set the teacher

The test-set dataset’s config is fully expanded, which means it names the relabeling teacher on its own in trace_processing.teacher_model_name. So the teacher that passed evaluation goes into both fields of the override. Download the config and set them:

distil dataset download-metadata -d train-set <test-dataset-id>
# in train-set/config.yaml:
#   base.teacher_model_name: <the teacher that passed>
#   trace_processing.teacher_model_name: <the same>
#   trace_processing.num_train_relabelled: 200
#   synthgen.train_generation_target: 10000

An override replaces the file whole, so edit the downloaded copy rather than writing a new one (Overrides).

Relabel traces into the train split

Run the expand on the test-set dataset. Skip it when no traces are left, or when you uploaded a train set and have no traces:

distil dataset relabel-traces-train --config train-set/config.yaml <test-dataset-id>
distil dataset status <train-dataset-id>

Generate synthetic train rows

Run generation on the dataset relabeling produced (or the test-set dataset when you skipped relabeling), smoke first. A smoke generates 128 rows as a side branch so you can check the data before generating all of it:

distil dataset generate-synthetic-data-train --smoke \
  --config train-set/config.yaml <train-dataset-id>

Read the smoke’s rows as described in Checking what came back, change one setting at a time until a smoke passes, then run the full job on the same parent, without --smoke:

distil dataset generate-synthetic-data-train --config train-set/config.yaml <train-dataset-id>

Check the train set

Download the final dataset and read the whole train split the same way: form against the job description, distribution against the relabeled rows and the traces, and the slices you asked for. Check the size against the plan. If it passes, the dataset goes to model training. If it doesn’t, don’t train on it: bad data shows up in training only as a score that stops improving, after a full training run. Fix the operation that produced the bad rows and run it again on its parent.

Topping up

A later iteration that needs more rows doesn’t rebuild. Run generation on the train split of the dataset your best model was trained on, with train_generation_target set to the rows to add:

distil dataset download-metadata -d top-up <best-dataset-id>
# in top-up/config.yaml: synthgen.train_generation_target: 2000, plus the new mutators
distil dataset generate-synthetic-data-train --config top-up/config.yaml <best-dataset-id>

The job generates from every row already in the split, relabeled and synthetic alike, and deduplicates against them, so the output is the old rows plus the new ones. Any traces left serve as context; after a full run there are usually none, and the top-up generates without context.

One exception: a train split that generation split into per-turn rows (clean_training_targets or enable_thinking on a multi-turn task) can’t be the examples of a later run. Top up on the dataset before that run instead, with the full target.

What can be changed

• The teacher. A different teacher is checked by teacher evaluation first; then both operations run again on the test-set dataset.
• The relabeled rows. Who rewrites them, with what guidance, how many (Relabeling). Generation imitates them, so better relabeling raises the whole set.
• The synthetic rows. Instructions, mutators, the synthgen config, or a top-up (Synthetic data generation).
• Hand-corrected rows. Download the dataset, fix train.jsonl, upload the directory as a new dataset with distil dataset create --data.

Cheapest first: a top-up or a hand correction (one credit), then generation again, then relabeling again (and generation after it), then a new teacher (teacher evaluation, then both).

Next

Model training.