Trace inputs

If you run an LLM in production, its logs are training data. This page covers the files a traces object is created from: your logs, the job description and the config. If you have no logs, an inference endpoint can collect them for you.

Pick your task type before you convert anything, since it decides the shape of everything else. See Task selection.

The input directory

Put the files in one directory. traces.jsonl holds your production logs, job_description.json describes the task in words, and config.yaml sets the task type, the models, the judge and the trace sections. test.jsonl is an optional curated test set.

traces-input/
├── traces.jsonl
├── job_description.json
├── config.yaml
└── test.jsonl

Upload the whole directory:

distil traces upload --data ./traces-input

The command prints Prepared traces created. ID: <traces-id>. Every later job on these traces takes that ID. The upload checks that the files are there and that the config parses. A malformed trace, test row or job description fails the first job that reads it, and that job’s log names the cause.

A curated test.jsonl stays in the test set, and Test set from traces adds rows to it.

Your traces

Your logs, one trace per line. Set trace_processing.observation_format in config.yaml to match their shape.

openai_messages is the default. Each line carries a messages array with roles system, user, assistant and tool. Assistant turns can carry tool_calls.

{"messages": [{"role": "system", "content": "..."}, {"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]}

openai_messages_with_images is the same, except user content can be a list of parts:

[
  {"type": "text", "text": "What is in this image?"},
  {"type": "image_url", "image_url": {"url": "..."}}
]

unstructured_with_openai_messages gives each line a context field: a string that contains the messages array as text.

langfuse is the format of an inference endpoint’s records when the traces object is created directly from the endpoint (Endpoint records to traces). You never write it by hand.

Converting your logs

• The system prompt goes into the job description. remove_system_prompt_from_traces (default true) strips leading system messages, so write the system prompt’s content into task_description. Variable input becomes the user message and model output becomes the assistant message.
• Every trace is a conversation, rewritten whole. A one-shot exchange is a two-turn conversation.
• Watch the length limit. synthgen.validation_max_total_length (default 30,000 characters) applies to processed rows, so raise it if your inputs embed documents or schemas.
• Write clean JSONL. Strip control characters and the unicode line separators U+2028 and U+2029, which are a common cause of parse failures in exported logs.

Job description

What your task is, in words. The teacher reads it when it relabels your traces, and again when it generates training data.

For a build from traces you usually want three fields:

{
  "task_description": "...",
  "llm_as_a_judge_instructions": "...",
  "trace_processing_instructions": "..."
}

• task_description describes the task. Write it from the system prompt your production system runs, with the same care: the output format with an example, the include and exclude rules, the edge cases. It stays the same through every iteration.

• llm_as_a_judge_instructions tells the judge what counts as a correct answer, including format rules such as no code fences. Optional, and not valid for classification, which is scored on label accuracy.

• trace_processing_instructions is added to the relabeling instructions only. Use it when the relabeling has to respect something unusual about your traces:

  This is a live phone call; preserve the caller’s interruptions and any cut-off utterances verbatim.

  Leave it out when no special handling is needed.

Classification also needs classes_description, and chat-completion-agentic needs tools (optional for chat-completion). The full field reference is Job description.

Config

The task type, the models, and the judge:

base:
  task: question-answering
  student_model_name: Qwen3.5-4B
  teacher_model_name: zai.glm-5.3-low-thinking
evaluation:
  llm_as_a_judge_model_name: zai.glm-5.3-low-thinking

trace_processing:
  observation_format: openai_messages

base.teacher_model_name is also the model that relabels your traces, unless trace_processing.teacher_model_name names a different one, so it matters from the first job. The large GLM 5.3 teacher, zai.glm-5.3-low-thinking, is the recommended choice (Supported models). Scores read by two different judges are not comparable, so the judge is usually set once for a project. Every other field has a default: Test set from traces and Trace processing cover the ones worth setting, and Config file has the full table.

Next

Build the test set from the traces.