Traces

If you run an LLM in production, its logs are training data. A dataset holds them as traces.jsonl, one conversation per line, next to your config and job description (Dataset overview). Relabeling turns traces into test and train rows, and synthetic data generation uses the rest as context (The expand operations). If you have no logs, an inference endpoint can collect them for you.

Pick your task type before you convert anything, since it decides the shape of everything else. See Task selection.

A trace file reaches the platform in one of three ways:

• distil dataset create --traces traces.jsonl next to --config and --job-description, and any train.jsonl or test.jsonl you have;
• distil traces upload --traces traces.jsonl, then distil dataset create-from-traces with the config and the job description;
• distil traces create-from-inference-endpoint, where the platform pulls an endpoint’s records and you write no file (Endpoint records to traces).

Your traces

Your logs, one trace per line. Set trace_processing.observation_format in config.yaml to match their shape.

openai_messages is the default. Each line carries a messages array with roles system, user, assistant and tool. Assistant turns can carry tool_calls.

{"messages": [{"role": "system", "content": "..."}, {"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]}

openai_messages_with_images is the same, except user content can be a list of parts:

[
  {"type": "text", "text": "What is in this image?"},
  {"type": "image_url", "image_url": {"url": "..."}}
]

unstructured_with_openai_messages gives each line a context field: a string that contains the messages array as text. Set only by Migrating old entities; you never write it by hand.

langfuse is the format of an inference endpoint’s records when the traces object is created directly from the endpoint (Endpoint records to traces). You never write it by hand; set it in the config of the dataset you create from that traces object.

Converting your logs

• The system prompt goes into the job description. Leading system messages are replaced by the job description’s prompt, so write the system prompt’s content into task_description. With base.variable_system_prompt on, each trace keeps its own system message instead. Variable input becomes the user message and model output becomes the assistant message.
• Every trace is a conversation, rewritten whole. A one-shot exchange is a two-turn conversation.
• Watch the length limit. synthgen.validation_max_total_length (default 30,000 characters) applies to relabeled rows, so raise it if your inputs embed documents or schemas.
• Write clean JSONL. Strip control characters and the unicode line separators U+2028 and U+2029, which are a common cause of parse failures in exported logs.

Job description

What your task is, in words. The teacher reads it when it relabels your traces, and again when it generates training data. For a build from traces you usually want three fields:

{
  "task_description": "...",
  "llm_as_a_judge_instructions": "...",
  "trace_processing_instructions": "..."
}

• task_description describes the task. Write it from the system prompt your production system runs, with the same care: the output format with an example, the include and exclude rules, the edge cases. It stays the same through every iteration.

• llm_as_a_judge_instructions tells the judge what counts as a correct answer, including format rules such as no code fences. Optional, and not valid for classification, which is scored on label accuracy.

• trace_processing_instructions is added to the relabeling instructions only. Use it when the relabeling has to respect something unusual about your traces:

  This is a live phone call; preserve the caller’s interruptions and any cut-off utterances verbatim.

  Leave it out when no special handling is needed.

Classification also needs classes_description, and chat-completion-agentic needs tools (optional for chat-completion). The full field reference is Job description.

Config

The task type, the models, and the judge:

base:
  task: question-answering
  student_model_name: Qwen3.5-4B
  teacher_model_name: zai.glm-5.3-low-thinking
evaluation:
  llm_as_a_judge_model_name: zai.glm-5.3-low-thinking

trace_processing:
  observation_format: openai_messages

base.teacher_model_name is also the model that relabels your traces, unless trace_processing.teacher_model_name names a different one, so it matters from the first job. The large GLM 5.3 teacher, zai.glm-5.3-low-thinking, is the recommended choice (Supported models). Scores read by two different judges are not comparable, so the judge is usually set once for a project. Every other field has a default: Relabeling and synthetic data covers the ones worth setting, and Config file has the full table.

Next

Create the dataset (Dataset overview), then build the test set.