How it works
Bring your agent's prompts and tools. Our agents build the data and environments from them, then auto-research loops train and evaluate your model until it beats the API you run today.
- 1
Start with the data you have
Your harness spec (system prompt, tools, schemas, policies), a few manual examples, and the traffic you are already cleared to use. Only the harness spec is required: no protected records, and no customer data before you choose to share it.
> Build a small model for our support agent from this repo: the harness spec in harness/, the 24 manual examples and the cleared traces in traces/. Go ahead. ⏺ Bash(distil dataset create --data support-agent/) ⎿ Upload successful. Dataset ID: ds-7f3a harness spec · 24 examples · 1,900 tracesYour agentAPI model
auto-research agentyour model v1agent changes · new traffic - 2
An auto-research agent builds your model
It builds a test set from your harness, simulates your agent's cases to generate training data including the rare ones, trains the model, evaluates it against the API you run today and repeats until it scores higher. Hours of training, minutes of your time.
> Train a model that beats the API we run today on the test set. Three runs at most, keep a log of every iteration. Go ahead. ⏺ Bash(distil dataset relabel-traces-test ds-7f3a) ⎿ ds-8c21 · 312 test conversations ⏺ Bash(distil dataset generate-synthetic-data-train ds-8c21) ⎿ ds-9d04 · 8,400 simulated conversations, 23% off script ⏺ Bash(distil slm create-from-dataset ds-9d04) ⎿ slm-2b7e · SFT, then RL · llm-as-a-judge 0.86 vs API today 0.88 ⏺ Run 1 misses topic switches. Run 2: more of them, 4 epochs. ⎿ slm-4c19 · llm-as-a-judge 0.91 vs API today 0.88 ✓ ahead
- 1Relabel traces
- 2Simulate conversations
- 3Post-train (SFT, then RL)
- 4Evaluate vs your API
- repeat until ahead
- 3
Deploy and evaluate
Deploy the model on distil labs inference, in your VPC or on-premise, and point your agent at it, with your current API as fallback. Score it on your own evals and on live traffic, side by side with that API, and keep it when it scores higher.
> Deploy slm-4c19 behind our support endpoint, current API as fallback. ⏺ Bash(distil deployment create-from-slm slm-4c19) ⎿ Deployment started. Deployment ID: dep-5a1c ⏺ Bash(distil inference-endpoint create-from-deployment --name support --fallback-model "openai/gpt-5" dep-5a1c) ⎿ Endpoint Name: support-Qk3bZ1 Endpoint: https://inference.distillabs.ai/v1/chat/completionsYour agentAPI model
auto-research agentyour model v1agent changes · new traffic - 4
Retrain the same day your agent changes
Your engineers rerun the same auto-research agent we use. The next version ships in about 80 minutes whenever your agent changes or a weak spot shows up, and once you have production traffic it flows back in.
> Two weeks of production traffic are cleared. Retrain on them. ⏺ Bash(distil traces create-from-inference-endpoint support-Qk3bZ1) ⎿ tr-c4e9 · 12,300 conversations since v1 ⏺ Bash(distil dataset create-from-traces --config config.yaml --job-description job_description.json tr-c4e9) ⎿ ds-a310 ⏺ Bash(distil slm create-from-dataset ds-a310) ⎿ slm-6e02 · v2 · SFT, then RL · about 80 minutesYour agentAPI model
auto-research agentyour model v1agent changes · new traffic
How it works
Bring your agent's prompts and tools. Our agents build the data and environments from them, then auto-research loops train and evaluate your model until it beats the API you run today.
Step 1 of 4
Start with the data you have
Your harness spec (system prompt, tools, schemas, policies), a few manual examples, and the traffic you are already cleared to use. Only the harness spec is required: no protected records, and no customer data before you choose to share it.
> Build a small model for our support agent from this repo: the
harness spec in harness/, the 24 manual examples and the cleared
traces in traces/. Go ahead.
⏺ Bash(distil dataset create --data support-agent/)
⎿ Upload successful. Dataset ID: ds-7f3a
harness spec · 24 examples · 1,900 tracesStep 2 of 4
An auto-research agent builds your model
It builds a test set from your harness, simulates your agent's cases to generate training data including the rare ones, trains the model, evaluates it against the API you run today and repeats until it scores higher. Hours of training, minutes of your time.
> Train a model that beats the API we run today on the test set. Three
runs at most, keep a log of every iteration. Go ahead.
⏺ Bash(distil dataset relabel-traces-test ds-7f3a)
⎿ ds-8c21 · 312 test conversations
⏺ Bash(distil dataset generate-synthetic-data-train ds-8c21)
⎿ ds-9d04 · 8,400 simulated conversations, 23% off script
⏺ Bash(distil slm create-from-dataset ds-9d04)
⎿ slm-2b7e · SFT, then RL · llm-as-a-judge 0.86 vs API today 0.88
⏺ Run 1 misses topic switches. Run 2: more of them, 4 epochs.
⎿ slm-4c19 · llm-as-a-judge 0.91 vs API today 0.88 ✓ aheadStep 3 of 4
Deploy and evaluate
Deploy the model on distil labs inference, in your VPC or on-premise, and point your agent at it, with your current API as fallback. Score it on your own evals and on live traffic, side by side with that API, and keep it when it scores higher.
> Deploy slm-4c19 behind our support endpoint, current API as fallback.
⏺ Bash(distil deployment create-from-slm slm-4c19)
⎿ Deployment started. Deployment ID: dep-5a1c
⏺ Bash(distil inference-endpoint create-from-deployment --name support
--fallback-model "openai/gpt-5" dep-5a1c)
⎿ Endpoint Name: support-Qk3bZ1
Endpoint: https://inference.distillabs.ai/v1/chat/completionsStep 4 of 4
Retrain the same day your agent changes
Your engineers rerun the same auto-research agent we use. The next version ships in about 80 minutes whenever your agent changes or a weak spot shows up, and once you have production traffic it flows back in.
> Two weeks of production traffic are cleared. Retrain on them.
⏺ Bash(distil traces create-from-inference-endpoint support-Qk3bZ1)
⎿ tr-c4e9 · 12,300 conversations since v1
⏺ Bash(distil dataset create-from-traces --config config.yaml
--job-description job_description.json tr-c4e9)
⎿ ds-a310
⏺ Bash(distil slm create-from-dataset ds-a310)
⎿ slm-6e02 · v2 · SFT, then RL · about 80 minutes
Get started with existing inputs

Your harness
- An endpoint to your conversational agent
- Your system prompt, tools and knowledge base
- We build mock APIs and test users around it

Traces
- Up to 10k real conversations, no labels needed
- Requests from a full day to size your deployment
- Or route n% of traffic through our endpoint

Evals
- The rubric or LLM judge you already trust
- Hand-labeled scenarios you came across
- None yet? We agree acceptance criteria together

Run the loop on your use case
Start on the platform today, or talk to us about your agent.