See what each agent costs, then replace the expensive ones with a hosted drop-in model at 50-90% lower inference cost. Launching with dltHub’s standardized ingestion layer, so there is no pipeline to build.
Your product runs agents. A few of them do the same narrow job thousands of times a day: classify a support ticket, extract fields from a document, turn a user request into a tool call. Each of those calls goes to a frontier model, and the invoice at the end of the month shows it.
The fix is not a secret. A small model fine-tuned on that one task does the same job at 50-90% lower inference cost, with lower latency. The training data already exists too; it is the traces your agents write every day.
The reason it rarely happens is the shape of that data. Traces are scattered across the tools that produce them, in more than thirty vendor formats that change with every release. Before anyone trains a model, someone has to build and maintain the extraction pipeline, and that is where the project usually dies.
Ingest, understand, replace
Today we are launching Agent Distillation, built with dltHub, to close that gap. The loop has three steps. dltHub’s verified sources ingest agent traces from wherever they already live and standardize them. Their Agent Cost & Usage package shows what every agent costs, per model, per person, per customer. Our Agent Distillation package turns the same standardized traces into the smaller model that replaces the expensive agent, trained and hosted by us. Ingest, understand, replace: one loop, and none of it is plumbing you build yourself.
On the dltHub side, verified sources cover Pydantic, Arize, and Langfuse at launch (LangChain coming soon), handle format drift, and normalize everything to OTel GenAI conventions. Agent Cost & Usage then breaks AI spend down by model, person, and customer, so deciding which agents are expensive stops being guesswork. The full ingestion and analytics story is in dltHub’s announcement.
The third step is ours. The agent_distillation transformation converts standardized traces into fine-tuning-ready data in OpenAI messages format, the input our platform consumes directly. Replacing an agent becomes a decision you make from the cost dashboard: pick a high-volume, well-scoped one, and its own traces are already in shape to train the replacement.
What you get: an endpoint, not an ML project
The deliverable is a hosted drop-in replacement. We host the model and serve it behind the same OpenAI-compatible API the expensive agent used, so switching means swapping one URL in existing code. Same behaviour on the task, with higher throughput, 50-90% lower inference cost, and lower latency. You write no training script, maintain no pipeline, and operate no inference infrastructure.
“The hard part for customers was never wanting a cheaper model, it was getting their trace data into a usable shape,” said Jacek Golebiowski, co-founder and CTO of distil labs. “With dltHub doing the ingestion and standardization, a customer goes from raw traces to a hosted drop-in endpoint with no script to write, no pipeline to maintain, and no inference to operate. We’ve seen a 1.7B specialist trained from traces this way beat a 744B frontier model on its target task, at 437 times smaller. The traces were always there; dltHub made them usable.”
“Nothing understands traces across vendors, and that’s the real blocker,” said Matthaus Krzykowski, CEO of dltHub. “We standardize the data so teams can finally see what their agents are doing, and distil labs lets them do something about it.”
How the replacement gets trained
distil labs fine-tunes task-specific small language models automatically: teams swap the general-purpose LLM in an agentic system for a smaller, purpose-built one that matches it on the narrow task. Setup takes under 30 minutes and the model is ready within 1-3 days.
The obvious objection is that production traces are noisy, drift with schema changes, and mix domains. We benchmarked exactly that, training on deliberately degraded traces: the fine-tuned model holds where direct training collapses. Full setup and datasets are in the traces-vs-synthetic benchmark.
We do not train on raw traces. The traces seed a teacher LLM that generates around 10,000 in-domain synthetic training examples; rule-based validators filter them for length, format, schema compliance, and novelty, which strips the teacher’s mistakes out of the training set; a compact open model is then fine-tuned on what survives and evaluated on a held-out test set against both the teacher and its own base model. This is also why the student can end up above its teacher: validation removes the teacher’s failures, and the small model spends all of its capacity on one task.
Agent Distillation is for narrow, recurring tasks that live inside a product; an open-ended coding or chat agent is not a candidate.
Getting started
The loop is available today. Point a dltHub verified source (Pydantic, Arize, or Langfuse) at your traces; the agent_distillation transformation lands a fine-tuning-ready dataset; onboard that dataset into distil labs, where we train, evaluate, and host the replacement model. The distillation step is priced by distil labs.
If you run an agent at meaningful spend, bring us that one agent: its system prompt and where its traces live. We will tell you within days whether a small model clears the bar on your task, with the evaluation to prove it. Start at distillabs.ai, or read dltHub’s announcement for the ingestion and analytics half of the loop.
distil labs trains task-specific small language models that are cheaper and faster per request than general-purpose LLMs, with equal-or-better accuracy on bounded tasks. Docs · GitHub · Slack