Model training

Training fine-tunes your student model on the generated dataset, then evaluates both the untrained and the trained student on your test set. That gives you the three-way comparison the deployment decision rests on.

This is the multi-hour GPU stage, so how much you experiment here is a budget decision.

Running it

distil slm create-from-training-dataset --output json <training-dataset-id> | jq -r .id
# <slm-id>

distil slm status --output json <slm-id> | jq -r .status

Each run spends one slms_from_training_datasets_post credit, and new accounts start with two. A run takes up to 90 minutes.

Wait for generation to finish first. A training dataset’s config only becomes readable at JOB_SUCCESS, so chaining submit-generation straight into read-config fails every time.

Configuring the training run

Training reads its baseline config from the training dataset, which inherited it from the seed dataset. Set sensible values before generation and the common case needs no override at all.

• base.student_model_name. The model you’re training. See Supported models, and note that tool-calling tasks restrict the family.
• tuning.per_device_train_batch_size (default 1). Higher is faster, but risks running out of memory.
• tuning.num_train_epochs (default 4).

When you raise the batch size, raise num_train_epochs with it. If credits don’t allow the extra epochs, prefer the smaller batch and the longer wall clock.

Training several students at once

A sweep is N submissions against the same dataset, one per student. The data is untouched, so it costs training credits only, and submissions run concurrently.

distil training-dataset download-metadata -d ./sweep <training-dataset-id>
# copy ./sweep/config.yaml once per student, changing base.student_model_name in each

for config in ./sweep/student-*.yaml; do
  distil slm create-from-training-dataset --output json --config "$config" \
    <training-dataset-id> | jq -r .id
done

Take a fresh copy per submission and send the config whole. A config trimmed to the fields you changed silently reverts everything you left out. See How the platform works.

Reading the results

distil slm metrics --output json <slm-id> \
  | jq '{base: .base_model_performance, tuned: .tuned_model_performance}'

distil slm download-predictions <slm-id>

You get three numbers on your task’s primary metric, and they only mean something together:

• base is the untrained student, the floor. It tells you how much of the task the student can already do before you spend anything.
• tuned is your trained student.
• Teacher is the ceiling, from teacher evaluation. The student learns from the teacher, so it can’t reliably exceed it.

What matters is how far tuned moved from base toward the teacher. A tuned of 0.72 is a good result against a teacher at 0.75 and a poor one against a teacher at 1.00, so read it as a position between the two rather than as a number on its own. For a sweep, one row per student, and the smallest student that gets close enough is usually the one to ship.

If you came from traces you have a fourth number: the original production model’s score, on base_model_performance of the seed dataset. That’s the model you’re replacing, so it’s the bar that actually matters in production.

Mind the noise band. Scores vary run to run, by a measured ±0.03 on a 50-row test set, so a difference smaller than that isn’t a difference. See Metrics.

If the student landed well short of the teacher, Improving your model covers what to do next.

You get both scores but only the tuned model’s predictions. The base-versus-tuned gap is available as numbers. A base-model failure case isn’t.

Retraining on the same data

Changing tuning parameters regenerates nothing:

distil training-dataset download-metadata -d ./retrain <training-dataset-id>
# in ./retrain/config.yaml, under tuning:
#   num_train_epochs: 6

distil slm create-from-training-dataset --output json \
  --config ./retrain/config.yaml <training-dataset-id> | jq -r .id

Dealing with out-of-memory failures

When a run crashes out of memory, apply these in order. Each costs more speed or quality than the one before. The first three are config changes that go through an ordinary override, and only the fourth touches the data.

1. Lower per_device_train_batch_size. Halve it, down to a floor of 1.
2. memory_optimized_training: true. Activation offloading plus gradient checkpointing. Significantly slower.
3. use_qlora: true together with memory_optimized_training: true. A 4-bit base model, roughly 3x less VRAM.
4. Filter out the longest ~1% of training examples. The long tail drives peak memory.

Search the log for the first error, not the last one. An out-of-memory crash often unwinds into a second, unrelated error, so the log can end with a pickling error thousands of characters after the OutOfMemoryError that actually caused it.

distil slm logs --output json <slm-id> | jq -r .logs

Next

Serve it, either hosted or on your own machine. If the result was short of the bar, see Improving your model.