Job description

job_description.json defines your task in words. The teacher reads it when it’s evaluated, the generator reads it when it writes training data, and the judge reads it when it scores a prediction.

This page is the full field reference. For what to put in it when you’re preparing a particular kind of input, see Trace inputs or Seed dataset overview.

Unknown fields are rejected at validation, so the fields below are the whole surface.

Fields by task type

Field	Classification	Question answering	Tool calling
task_description	required	required	required
classes_description	required	-	-
tools	-	-	required
llm_as_a_judge_instructions	rejected	optional	optional
synthetic_data_generation_instructions	optional	optional	optional
trace_processing_instructions	optional	optional	optional

The question answering column covers question-answering, question-answering-open-book and question-answering-closed-book. The tool calling column covers tool-calling-closed-book and multi-turn-tool-calling-closed-book.

llm_as_a_judge_instructions isn’t valid for classification, and including it fails the create. Classification is judged by label accuracy, so there’s no judge to instruct.

What each field feeds

task_description

Goes into every teacher prompt: evaluation, generation and judging. This is the single most load-bearing field in the file.

Derive it from the system prompt of the system you’re replacing, with the same care: the output format with an example, the include and exclude rules, the edge cases.

It has to stay constant across iterations. task_description isn’t a tuning lever: when results are poor, pick a better teacher or fix the data instead. Changing it between attempts means you’re no longer comparing like with like, and because it also feeds the judge, changing it changes what “correct” means everywhere at once.

classes_description

Maps each class name to a description of when it applies.

{
  "classes_description": {
    "billing": "questions about invoices, charges, refunds or payment methods",
    "technical": "the product is not working as documented"
  }
}

These descriptions define the label space and directly shape the data that gets generated, so a vague class description produces vague examples of that class.

The label sets in train.jsonl, test.jsonl and classes_description have to be identical, or validation fails.

tools

A list of tool schemas in the OpenAI function-calling format: at least one tool, unique names.

{
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Look up the current weather for a city",
        "parameters": {
          "type": "object",
          "properties": { "city": { "type": "string" } },
          "required": ["city"]
        }
      }
    }
  ]
}

Parameter schemas can declare default values. The tool_call_equivalence metric treats an argument left at its default as equal to omitting it.

Every tool call in your training and test data validates against these schemas.

llm_as_a_judge_instructions

What the judge is told when it scores a prediction against the reference. State the pass and fail criteria: what has to match, and what to ignore, such as ordering, whitespace or paraphrasing.

A shape that works:

Output ‘good’ if the prediction matches the reference or is semantically equivalent, otherwise output ‘bad’.

followed by the criteria that decide it.

Vague criteria make every downstream number noisy. You compare teacher, base and tuned scores from this judge, so noise here propagates into every comparison you make afterwards. The judge sees the full prefix, including the question and any context.

synthetic_data_generation_instructions

Extra guidance injected into every generation prompt, and read during generation only. Use it to describe what the generated inputs should look like: formats, domains, register, noise.

Because it touches generation and nothing else, it’s the safe place to steer the inputs without redefining the task. See Synthetic data generation.

trace_processing_instructions

Task-specific guidance appended to the trace-processing rewrite and fix instructions. It does nothing outside trace processing, so leave it out unless you’re starting from traces.

Use it when the edits have to respect something unusual about your traces:

This is a live phone call; preserve the caller’s interruptions and any cut-off utterances verbatim.

See Trace processing.