How it works

Bring your agent's prompts and tools. Our agents build the data and environments from them, then auto-research loops train and evaluate your model until it beats the API you run today.

  1. 1

    Start with the data you have

    Your harness spec (system prompt, tools, schemas, policies), a few manual examples, and the traffic you are already cleared to use. Only the harness spec is required: no protected records, and no customer data before you choose to share it.

    > Build a small model for our support agent from this repo: the
      harness spec in harness/, the 24 manual examples and the cleared
      traces in traces/. Go ahead.
    
    ⏺ Bash(distil dataset create --data support-agent/)
      ⎿  Upload successful. Dataset ID: ds-7f3a
         harness spec · 24 examples · 1,900 traces
  2. 2

    An auto-research agent builds your model

    It builds a test set from your harness, simulates your agent's cases to generate training data including the rare ones, trains the model, evaluates it against the API you run today and repeats until it scores higher. Hours of training, minutes of your time.

    > Train a model that beats the API we run today on the test set. Three
      runs at most, keep a log of every iteration. Go ahead.
    
    ⏺ Bash(distil dataset relabel-traces-test ds-7f3a)
      ⎿  ds-8c21 · 312 test conversations
    ⏺ Bash(distil dataset generate-synthetic-data-train ds-8c21)
      ⎿  ds-9d04 · 8,400 simulated conversations, 23% off script
    ⏺ Bash(distil slm create-from-dataset ds-9d04)
      ⎿  slm-2b7e · SFT, then RL · llm-as-a-judge 0.86 vs API today 0.88
    ⏺ Run 1 misses topic switches. Run 2: more of them, 4 epochs.
      ⎿  slm-4c19 · llm-as-a-judge 0.91 vs API today 0.88 ✓ ahead
    1. 1Relabel traces
    2. 2Simulate conversations
    3. 3Post-train (SFT, then RL)
    4. 4Evaluate vs your API
    5. repeat until ahead
  3. 3

    Deploy and evaluate

    Deploy the model on distil labs inference, in your VPC or on-premise, and point your agent at it, with your current API as fallback. Score it on your own evals and on live traffic, side by side with that API, and keep it when it scores higher.

    > Deploy slm-4c19 behind our support endpoint, current API as fallback.
    
    ⏺ Bash(distil deployment create-from-slm slm-4c19)
      ⎿  Deployment started. Deployment ID: dep-5a1c
    ⏺ Bash(distil inference-endpoint create-from-deployment --name support
           --fallback-model "openai/gpt-5" dep-5a1c)
      ⎿  Endpoint Name: support-Qk3bZ1
         Endpoint:      https://inference.distillabs.ai/v1/chat/completions
  4. 4

    Retrain the same day your agent changes

    Your engineers rerun the same auto-research agent we use. The next version ships in about 80 minutes whenever your agent changes or a weak spot shows up, and once you have production traffic it flows back in.

    > Two weeks of production traffic are cleared. Retrain on them.
    
    ⏺ Bash(distil traces create-from-inference-endpoint support-Qk3bZ1)
      ⎿  tr-c4e9 · 12,300 conversations since v1
    ⏺ Bash(distil dataset create-from-traces --config config.yaml
           --job-description job_description.json tr-c4e9)
      ⎿  ds-a310
    ⏺ Bash(distil slm create-from-dataset ds-a310)
      ⎿  slm-6e02 · v2 · SFT, then RL · about 80 minutes

How it works

Bring your agent's prompts and tools. Our agents build the data and environments from them, then auto-research loops train and evaluate your model until it beats the API you run today.

Step 1 of 4

Start with the data you have

Your harness spec (system prompt, tools, schemas, policies), a few manual examples, and the traffic you are already cleared to use. Only the harness spec is required: no protected records, and no customer data before you choose to share it.

> Build a small model for our support agent from this repo: the
  harness spec in harness/, the 24 manual examples and the cleared
  traces in traces/. Go ahead.

⏺ Bash(distil dataset create --data support-agent/)
  ⎿  Upload successful. Dataset ID: ds-7f3a
     harness spec · 24 examples · 1,900 traces

Step 2 of 4

An auto-research agent builds your model

It builds a test set from your harness, simulates your agent's cases to generate training data including the rare ones, trains the model, evaluates it against the API you run today and repeats until it scores higher. Hours of training, minutes of your time.

> Train a model that beats the API we run today on the test set. Three
  runs at most, keep a log of every iteration. Go ahead.

⏺ Bash(distil dataset relabel-traces-test ds-7f3a)
  ⎿  ds-8c21 · 312 test conversations
⏺ Bash(distil dataset generate-synthetic-data-train ds-8c21)
  ⎿  ds-9d04 · 8,400 simulated conversations, 23% off script
⏺ Bash(distil slm create-from-dataset ds-9d04)
  ⎿  slm-2b7e · SFT, then RL · llm-as-a-judge 0.86 vs API today 0.88
⏺ Run 1 misses topic switches. Run 2: more of them, 4 epochs.
  ⎿  slm-4c19 · llm-as-a-judge 0.91 vs API today 0.88 ✓ ahead

Step 3 of 4

Deploy and evaluate

Deploy the model on distil labs inference, in your VPC or on-premise, and point your agent at it, with your current API as fallback. Score it on your own evals and on live traffic, side by side with that API, and keep it when it scores higher.

> Deploy slm-4c19 behind our support endpoint, current API as fallback.

⏺ Bash(distil deployment create-from-slm slm-4c19)
  ⎿  Deployment started. Deployment ID: dep-5a1c
⏺ Bash(distil inference-endpoint create-from-deployment --name support
       --fallback-model "openai/gpt-5" dep-5a1c)
  ⎿  Endpoint Name: support-Qk3bZ1
     Endpoint:      https://inference.distillabs.ai/v1/chat/completions

Step 4 of 4

Retrain the same day your agent changes

Your engineers rerun the same auto-research agent we use. The next version ships in about 80 minutes whenever your agent changes or a weak spot shows up, and once you have production traffic it flows back in.

> Two weeks of production traffic are cleared. Retrain on them.

⏺ Bash(distil traces create-from-inference-endpoint support-Qk3bZ1)
  ⎿  tr-c4e9 · 12,300 conversations since v1
⏺ Bash(distil dataset create-from-traces --config config.yaml
       --job-description job_description.json tr-c4e9)
  ⎿  ds-a310
⏺ Bash(distil slm create-from-dataset ds-a310)
  ⎿  slm-6e02 · v2 · SFT, then RL · about 80 minutes

Get started with existing inputs

Required

Your harness

  • An endpoint to your conversational agent
  • Your system prompt, tools and knowledge base
  • We build mock APIs and test users around it
Helpful

Traces

  • Up to 10k real conversations, no labels needed
  • Requests from a full day to size your deployment
  • Or route n% of traffic through our endpoint
Helpful

Evals

  • The rubric or LLM judge you already trust
  • Hand-labeled scenarios you came across
  • None yet? We agree acceptance criteria together

Run the loop on your use case

Start on the platform today, or talk to us about your agent.