System prompts

By default a model has one system prompt, built from the job description and used in every training row, every evaluation row and every request. One config key changes that: base.variable_system_prompt (default false). With it on, every row keeps its own system message, synthetic data generation writes a different system message for every example, and the trained model follows the system message each request sends. It works for every task type.

When to use it

Keep the default unless you need it. A fixed system prompt is the simpler setup: the model learns one task, and nothing a request sends changes its behaviour.

Turn it on when one model serves several applications, customers or policies whose instructions differ: a rule that changes the answer, a default value, a forbidden option, a term to resolve, a different output format. Without it, each variant needs its own model.

The fixed system prompt

With the flag off, every row and every request uses the job description’s system prompt:

Task	System prompt
question-answering, chat-completion, chat-completion-agentic	task_description
classification	task_description, followed by the list of classes from classes_description

With the flag on, a row without a system message gets this same prompt.

Where to set it

Set base.variable_system_prompt: true in the config.yaml of your dataset, before the first expand, so relabeling, generation, teacher evaluation and training all read it:

base:
  task: question-answering
  variable_system_prompt: true

Stage	Flag on	Flag off
Loading train and test rows	A row keeps its system message. A row without one gets the fixed prompt	Every row gets the fixed prompt; system messages in the files are replaced
Relabeling traces	A trace keeps its system message	A trace gets the fixed prompt
Synthetic data generation	The teacher writes a system message for every new example	Every new example gets the fixed prompt
Teacher evaluation	The teacher answers each test row with that row’s system message	The teacher answers with the fixed prompt
Training	The student trains on each row’s system message	The student trains on the fixed prompt

trace_processing.remove_system_prompt_from_traces has no effect; this key decides.

System prompts in your data

Write the system message as the first message of a row, exactly as your application will send it:

{"messages": [{"role": "system", "content": "Do not redact person names; agents need the caller's name to verify the account."}, {"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]}

• One system message per row, at the start. Rows without one are allowed and get the fixed prompt.
• For classification, a row’s own system message must list the classes itself: the class list is added only to the fixed prompt.
• Test rows carry the system messages your application will send, so the test set measures whether the model follows them.
• Traces in openai_messages format keep the system message they were recorded with.

Writing the job description

With the flag on, task_description has two jobs: it describes the task to the teacher in relabeling, generation, evaluation and judging, and it is the system prompt of rows without one. Write it as a short statement of what every application has in common, not as one application’s full prompt. The system messages hold what differs between applications.

• For a tool-calling task this can be one sentence: Control a smart home: turn the user's commands into calls of the available tools.
• For a task with defaults that system messages override, task_description states the defaults, for example the redaction tokens, and the system messages state the deviations.

A copy of one application’s full prompt in task_description is copied by the teacher into the generated examples, and most generated rows then carry that same prompt.

Generating system messages

Without guidance the teacher writes system messages that paraphrase the task description. The model then learns that the system message doesn’t matter. Steer the system messages with a mutator whose values describe a system message that changes the correct answer, each ending with “without the system message the answer would be wrong”:

synthgen:
  mutators:
    - name: system prompt
      values:
        - "written for an application whose system message states a rule that makes the correct answer a different tool call or different arguments than it would be without the rule; the answer obeys the rule, and without the system message the answer would be wrong"
        - "written for an application whose system message sets a default value the user never mentions; the correct tool call carries that value only because of the system message, and without the system message the answer would be wrong"
        - "written for an application whose system message forbids one tool or one option and names the replacement; the user asks for the forbidden one and the answer uses the replacement, and without the system message the answer would be wrong"
        - "written for an application whose system message defines a nickname or term that the user then uses; the answer resolves it as defined, and without the system message the answer would be wrong"
        - "written for an application whose system message restricts what the assistant may do; the user asks for something outside the restriction and the answer stays within it, and without the system message the answer would be wrong"
    - name: household
      values:
        - "set in a small studio apartment with a single room"
        - "set in a large family house with many rooms and a garage"
        - "set in a shared flat where several people give commands"

• Write the values for your task: for a redaction task, a changed token format, an exempt category, an added category, a changed masking rule.
• The closing clause matters. Milder values, such as “states a rule the answer must follow”, produce system messages that most answers ignore.
• A second mutator varies the content, so the examples differ in more than their system message.
• A generated example without a system message is dropped by validation.

Sample system messages written with these values:

The housekeeper's checklist calls the thermostat 'the climate box'.
The kiosk is out of tomato sauce today. Only alfredo sauce is available; use it whenever a sauce is needed.

Evaluation

The teacher answers each test row with that row’s system message, and the LLM judge sees the system message as part of the input.

To check that the answers depend on the system messages, run two teacher evaluations on the same test set: one as configured, one with base.variable_system_prompt: false as a config override, which replaces every system message by the fixed prompt. If the two scores are close, the system messages don’t change the answers: rewrite the test rows or the mutator values before you build the train set. Read the predictions of the rows the teacher gets right only with their own system message to see which kinds of system message matter.

Deployment

Deployment doesn’t change. The model must run with a system prompt in every request: the task_description it was trained with, or, with variable_system_prompt on, the system prompt your application chooses for that request (Hosted inference).

Next

Every other parameter: Config file. The stage that writes the system messages: Relabeling and synthetic data.