Overview

Building a model on distil labs is a straight line. Your traces and your labeled rows live in one dataset. Expand operations grow it: relabeled traces and synthetic rows into the test set, then, with the teacher that scored best on that test set, into the train set. The student is trained on the result and goes behind an inference endpoint that serves your traffic.

 A  LLM in production ──►  1  Inference endpoint, collect traces
                              │
                           2  Traces object ─── from the endpoint's records
                              │
 B  your files ──────────►  3  Dataset
    (any of traces,           │   A: from the traces object
     train, test)             │   B: uploaded; missing splits stay empty
                              │
                           4  Build the test set ─── relabel, then synthetic rows
                              │
                           5  Teacher evaluation ─── pick the teacher, score the production model
                              │
                           6  Build the train set ─── relabel, then synthetic rows
                              │
                           7  Model training
                              │
                           8  Serve behind an endpoint

Where you start

You have	Start at
An LLM in production that OpenRouter serves, and you can point your application at an inference endpoint	Inference endpoint
Files: a trace file (exported logs, converted), a labeled train set, a test set, or any combination	Dataset

Both entries meet at the dataset. An inference endpoint’s fallback is an OpenRouter model, and the last step serves your model behind an endpoint, so when your production model is on OpenRouter, starting with the endpoint uses the same endpoint setup at both ends.

The dataset

One entity holds the data: the dataset, with config.yaml, job_description.json, traces.jsonl, train.jsonl and test.jsonl. Any data file can be empty. A dataset is uploaded, or created from a traces object, or produced from another dataset by one of four expand operations: relabel traces into the test or the train split, or generate synthetic rows into the test or the train split. Each expand is a new dataset whose parent is the previous one, so a project is a chain of datasets, and a teacher evaluation or a trained model is made from any dataset in it (How the platform works).

The stages

Stage	What it does
Task selection	Picks the task type, which sets the shape of your data
Inference endpoint	Puts an endpoint in front of the model you run today
Collect traces	Records your production traffic
Endpoint records to traces	Turns the records into a dataset
Dataset	Your files as a dataset
Building the test set	Relabels traces into test rows, adds synthetic rows if needed
Teacher evaluation	Scores each candidate teacher and your production model on the test set
Building the train set	Relabels traces into train rows, then the teacher writes the training data
Model training	Fine-tunes and evaluates the student
Serving behind the endpoint	Puts the student in front of your traffic
Model iterations	Improves a model that isn’t good enough yet

Relabeling and synthetic data describes the two expand operations the test-set and train-set stages are built from. How the jobs, entities and credits behind these stages work is in How the platform works. The agent skill runs this loop for you and decides what to run next.

The two files every entry needs

Every dataset needs two files, written before you create it:

• config.yaml sets the task type and the models.
• job_description.json describes the task the model learns.

An example config. Every field not shown has a default; Config file lists them all:

base:
  task: question-answering
  student_model_name: Qwen3.5-4B
  teacher_model_name: zai.glm-5.3-low-thinking
evaluation:
  llm_as_a_judge_model_name: zai.glm-5.3-low-thinking

An example job description for question answering. The fields for the other task types are in Job description:

{
  "task_description": "Answer the customer's question about their order in one or two sentences, using only the order details in the message."
}