Build your own models instead of renting them
distil labs turns your harness and a few examples into agent-specific models that beat the APIs you run today on accuracy and latency, built in hours.
30M+ people use models trained on distil labs today
No off-the-shelf model fits your agent
The API your agent runs today caps its quality, and every competitor on the same API ships the same agent. Efficient APIs cost less and lose accuracy. Frontier APIs are accurate, but priced per call and never yours. The model your agent needs has to be built, and in-house that takes a team and months. With distil labs, your team ships it in hours.
How it works
- 1
Start with the data you have
Your harness spec (system prompt, tools, schemas, policies), a few manual examples, and the traffic you are already cleared to use. Only the harness spec is required: no protected records, and no customer data before you choose to share it.
> Build a small model for our support agent from this repo: the harness spec in harness/, the 24 manual examples and the cleared traces in traces/. Go ahead. ⏺ Bash(distil dataset create --data support-agent/) ⎿ Upload successful. Dataset ID: ds-7f3a harness spec · 24 examples · 1,900 tracesYour agentAPI modelauto-research agentyour model v1agent changes · new traffic - 2
An auto-research agent builds your model
It builds a test set from your harness, simulates your agent's cases to generate training data including the rare ones, trains the model, evaluates it against the API you run today and repeats until it scores higher. Hours of training, minutes of your time.
> Train a model that beats the API we run today on the test set. Three runs at most, keep a log of every iteration. Go ahead. ⏺ Bash(distil dataset relabel-traces-test ds-7f3a) ⎿ ds-8c21 · 312 test conversations ⏺ Bash(distil dataset generate-synthetic-data-train ds-8c21) ⎿ ds-9d04 · 8,400 simulated conversations, 23% off script ⏺ Bash(distil slm create-from-dataset ds-9d04) ⎿ slm-2b7e · SFT, then RL · llm-as-a-judge 0.86 vs API today 0.88 ⏺ Run 1 misses topic switches. Run 2: more of them, 4 epochs. ⎿ slm-4c19 · llm-as-a-judge 0.91 vs API today 0.88 ✓ ahead- 1Relabel traces
- 2Simulate conversations
- 3Post-train (SFT, then RL)
- 4Evaluate vs your API
- repeat until ahead
- 3
Deploy and evaluate
Deploy the model on distil labs inference, in your VPC or on-premise, and point your agent at it, with your current API as fallback. Score it on your own evals and on live traffic, side by side with that API, and keep it when it scores higher.
> Deploy slm-4c19 behind our support endpoint, current API as fallback. ⏺ Bash(distil deployment create-from-slm slm-4c19) ⎿ Deployment started. Deployment ID: dep-5a1c ⏺ Bash(distil inference-endpoint create-from-deployment --name support --fallback-model "openai/gpt-5" dep-5a1c) ⎿ Endpoint Name: support-Qk3bZ1 Endpoint: https://inference.distillabs.ai/v1/chat/completionsYour agentAPI modelauto-research agentyour model v1agent changes · new traffic - 4
Retrain the same day your agent changes
Your engineers rerun the same auto-research agent we use. The next version ships in about 80 minutes whenever your agent changes or a weak spot shows up, and once you have production traffic it flows back in.
> Two weeks of production traffic are cleared. Retrain on them. ⏺ Bash(distil traces create-from-inference-endpoint support-Qk3bZ1) ⎿ tr-c4e9 · 12,300 conversations since v1 ⏺ Bash(distil dataset create-from-traces --config config.yaml --job-description job_description.json tr-c4e9) ⎿ ds-a310 ⏺ Bash(distil slm create-from-dataset ds-a310) ⎿ slm-6e02 · v2 · SFT, then RL · about 80 minutesYour agentAPI modelauto-research agentyour model v1agent changes · new traffic
How it works
Step 1 of 4
Start with the data you have
Your harness spec (system prompt, tools, schemas, policies), a few manual examples, and the traffic you are already cleared to use. Only the harness spec is required: no protected records, and no customer data before you choose to share it.
> Build a small model for our support agent from this repo: the
harness spec in harness/, the 24 manual examples and the cleared
traces in traces/. Go ahead.
⏺ Bash(distil dataset create --data support-agent/)
⎿ Upload successful. Dataset ID: ds-7f3a
harness spec · 24 examples · 1,900 tracesStep 2 of 4
An auto-research agent builds your model
It builds a test set from your harness, simulates your agent's cases to generate training data including the rare ones, trains the model, evaluates it against the API you run today and repeats until it scores higher. Hours of training, minutes of your time.
> Train a model that beats the API we run today on the test set. Three
runs at most, keep a log of every iteration. Go ahead.
⏺ Bash(distil dataset relabel-traces-test ds-7f3a)
⎿ ds-8c21 · 312 test conversations
⏺ Bash(distil dataset generate-synthetic-data-train ds-8c21)
⎿ ds-9d04 · 8,400 simulated conversations, 23% off script
⏺ Bash(distil slm create-from-dataset ds-9d04)
⎿ slm-2b7e · SFT, then RL · llm-as-a-judge 0.86 vs API today 0.88
⏺ Run 1 misses topic switches. Run 2: more of them, 4 epochs.
⎿ slm-4c19 · llm-as-a-judge 0.91 vs API today 0.88 ✓ aheadStep 3 of 4
Deploy and evaluate
Deploy the model on distil labs inference, in your VPC or on-premise, and point your agent at it, with your current API as fallback. Score it on your own evals and on live traffic, side by side with that API, and keep it when it scores higher.
> Deploy slm-4c19 behind our support endpoint, current API as fallback.
⏺ Bash(distil deployment create-from-slm slm-4c19)
⎿ Deployment started. Deployment ID: dep-5a1c
⏺ Bash(distil inference-endpoint create-from-deployment --name support
--fallback-model "openai/gpt-5" dep-5a1c)
⎿ Endpoint Name: support-Qk3bZ1
Endpoint: https://inference.distillabs.ai/v1/chat/completionsStep 4 of 4
Retrain the same day your agent changes
Your engineers rerun the same auto-research agent we use. The next version ships in about 80 minutes whenever your agent changes or a weak spot shows up, and once you have production traffic it flows back in.
> Two weeks of production traffic are cleared. Retrain on them.
⏺ Bash(distil traces create-from-inference-endpoint support-Qk3bZ1)
⎿ tr-c4e9 · 12,300 conversations since v1
⏺ Bash(distil dataset create-from-traces --config config.yaml
--job-description job_description.json tr-c4e9)
⎿ ds-a310
⏺ Bash(distil slm create-from-dataset ds-a310)
⎿ slm-6e02 · v2 · SFT, then RL · about 80 minutesWhat you get
Build your own model in hours, without post-training experience
An auto-research agent turns your harness and examples into curated training data and delivers the first model within hours, then retrains in about 80 minutes, so you change your agent and test the new model the same day.
Nobody on your team needs to have post-trained a model before.
- In-house team or agency, per agentmonths
- Your first model with distil labshours
- Each retrain after thatabout 80 minutes
More accurate than your API, at the same cost
Your model scores higher than the API you run today on your own evals, including the rare cases production traffic barely contains, because we generate training data for uncommon situations and for each failure you describe.
Logs teach a model the cases that went well. Simulated data covers the ones that go off script.
Intelligence your competitors cannot buy
Every model you build turns your harness and data into a model you own. Agents on the same API are the same agent; yours is the one that stands out.
Your engineers run the same auto-research agent and tools we use, so after the first model your team builds and retrains every model itself.
What the alternatives cannot do
| Larger API model | Post-training agency | distil labs | |
|---|---|---|---|
| High quality even on rare conversations | quality goes up | after months | on your own evals |
| Same cost and latency as today | costs more, slower | depends on the model | smaller model, lower latency |
| First model in hours | nothing owned | months per project | hours of training |
| Expertise stays in-house | you own nothing | stays with their engineers | your engineers run the same agent |
| Start without customer data | nothing to train | needs production traces | harness and a few examples |
Ship your first agent-specific model in hours
Bring your harness and a few examples. We show you a model that beats the API you run today on your own evals, including the difficult cases, before any customer data is shared.
Wondering what happens to your harness? Security and data handling
What Our Customers Say
We needed a small model that could power our product on an IBM P11, entirely on-premises. distil labs’ fine-tuned models allowed us to ship a self-contained solution where the SLM and our graph platform coexist on the same hardware. For customers in regulated industries, this means AI-powered query generation with complete data privacy – nothing ever leaves their environment.
David J. Haglin
Co-Founder and CTO at Rocketgraph
The collaboration with distil labs shows what’s possible when you fine-tune small language models and deploy them on IBM Power. It is not just about the performance–it’s about delivering private, accurate AI that’s ready for enterprise scale.
Unnikrishnan Rajagopal
Director at IBM Power
The distil labs platform accelerated the release of our cybersecurity-specialized language model, KINDI, enabling faster iterations with greater confidence. As a result, we ship InovaGuard improvements sooner and continuously boost investigation accuracy with every release.
Samir Bennacer
Co-Founder and CTO at Octodet
From our blog
How Knowunity used distil labs to cut their LLM bill by 68%
Knowunity, an edtech startup processing hundreds of millions of AI requests monthly, used distil labs to train a custom small language model that cut inference costs by 68% while improving classification accuracy from 81% to 93%.
Why training on production traces fails (and what to do instead)
Training directly on production traces doesn't work as well as you'd expect. We tested across five scenarios and synthetic data from traces scores up to 26 percentage points higher in accuracy.
The 10x Inference Tax You Don't Have to Pay
Benchmarking fine-tuned small language models (0.6B-8B) against 10 frontier LLMs across 8 datasets shows that task-specific SLMs match or beat frontier models at 10-100x lower inference cost.