Frequently Asked Questions

distil labs is a developer platform that turns your production traces into a small model that runs the same task far cheaper. You train a model with your current system prompt and a few hundred of your agent traces, and from those traces we generate synthetic data and fine-tune a small student model.

See how the platform works.

SLM means Small Language Model. There is no universal cutoff, but our current models range from 135M to around 9B parameters. Unlike a frontier model designed to handle almost anything, an SLM is a specialist trained to do one task very well.

We are a product company. You can train models through our CLI without building your own training infrastructure.

We sit a layer above the training kernels, not against them. Unsloth, HF AutoTrain and Tinker are hands-on libraries you drive yourself: you still need to bring the dataset, pick the teacher and student, design the eval, and run the loop. We automate that surrounding loop, generating synthetic data from your job description, distilling from a teacher, and running an eval harness. The output is a standard open-weight model you own. If you would rather own the dataset and the eval yourself, those libraries are the right tool.

Three come up again and again with teams doing it themselves. Getting enough clean training data, because gathering and labeling blocks the start. Evaluation, because you have to prove the model actually improved rather than overfit. And maintenance, because schemas and prompts change, and keeping self-built models updated is a job of its own. Those three are exactly what the platform takes off you: it generates the data, runs the eval, and makes retraining a re-run of the same pipeline rather than a rebuild.

Yes, all open-weight. The students come from our supported models catalog and run from 135M to around 9B: Qwen3 and Qwen3.5, Llama 3, Gemma 3 and Gemma 4, SmolLM2, and Liquid LFM2 and LFM2.5. See supported models.

Start with Qwen3 1.7B, read the metrics, and only move up in size if the results fall short. Beyond that, your serving constraints decide it. Students under 1B are the fastest and cheapest to run and fit on-device, but need more training data to reach a given quality bar. From 3B up you usually reach higher quality with less work, at the cost of slower and more expensive inference. In practice the eval settles it: train two sizes on the same data and compare them on your test set. One caveat: tool calling works with a subset of the catalog, so that narrows the choice. See supported models.

No, the teachers are open-weight too. The default is GPT OSS 120B, and the catalog also carries DeepSeek, Qwen3, GLM, Kimi and MiniMax, so your training data is never generated by a closed model. Teachers are a separate list from students, and not every teacher handles tool calling. See supported models.

Two paths. If you have production traces, a few hundred plus your system prompt is plenty: the pipeline uses at most 200 as training seeds and another 200 for the test set, and any surplus becomes domain context. If you don’t, a minimal dataset works: 20+ diverse labeled examples in the conversation format your tooling already produces, plus a held-out test set. Either way, the platform generates around 10,000 synthetic training examples from that seed, so you never label thousands yourself. See data preparation.

Up to 200 of your traces become relabelled training seeds, and the surplus goes in as domain context. A large open teacher model reads them and generates around 10,000 synthetic examples for your task, and each one is validated and filtered before it enters the training set. The student is then fine-tuned on that curated set: plain supervised fine-tuning on generated input-output pairs. No logits or reasoning traces are needed, so any teacher in the catalog works for text tasks, though tool calling narrows the list. Training on the curated set instead of the raw traces is what makes the model better. Read the benchmark.

LoRA by default. QLoRA and full fine-tuning are both available if you set them in the config. For a narrow task you’re teaching the model a specific behaviour rather than new general knowledge, and an adapter does that well while training far faster and cheaper. You download the trained model when the run finishes and either deploy it on a managed endpoint or run it yourself.

Not today. The pipeline is supervised fine-tuning on synthetic data generated from your traces. We did benchmark RLVR on top of SFT across 12 datasets: it improves text generation by around 2 points and gives no reliable benefit on structured tasks like classification and tool calling. Read the benchmark.

We test the teacher before we train the student. Teacher evaluation is its own step: if the teacher can’t solve your task on your examples, you find out before spending hours on training. After that, every synthetic example is validated and filtered, and the student is scored against your held-out test set, so you see the accuracy before you deploy.

For an occasional change, it is light: the data generation and eval are automated, so updating a model for a new schema is re-running the same pipeline rather than starting over. You update the job description and tool schema, retrain, and compare the eval against the previous run. Each retrain costs a training credit, so if your schema changes weekly the cost adds up. See pricing.

We support six task types: classification, question answering, tool calling, multi-turn tool calling, open book QA (RAG), and closed book QA. In product terms, that covers chatbots, voice agents, structured output extraction, intent detection, ticket triage, content moderation, pulling fields out of contracts or invoices, and routing requests to APIs. Within those, the best fit is a narrow, repeatable, high-volume task, and the higher the volume the bigger the saving. See task selection.

Yes, within limits. Tool calling and multi-turn tool calling are two of the six task types, so the model learns your tool schema during training instead of looking it up on every call. The ceiling is your student’s context window, because the whole tool schema travels with every call: a thousand-tool agent is not something we can do today. In one of our benchmarks, a tuned 1.7B student beat its own 744B teacher on multi-turn tool calling. Tool calling narrows both catalogs, so check that your student and your teacher both support it. Read the write-up.

Yes, that is what open book QA is for: the model answers strictly from the chunks your retriever passes it. If retrieval itself is the hard part, closed book QA is the other option, where the model learns your knowledge during training and answers with no context at all. Keep a frontier model for the open-ended reasoning steps. In practice it is a blend rather than a full swap.

Not on the self-serve platform. The supported models catalog is text-only today, so there is no vision student you can pick and train yourself.

We do train vision-language models, and in practice that covers question answering over images and OCR-style extraction from scanned documents. Two limits worth knowing: image inputs are scoped to QA tasks rather than the full set, and PDFs are not handled natively, so pass them as text or as rendered images. We don’t do classic computer-vision tasks like object detection or segmentation.

If you have a task in mind, reach out at contact@distillabs.ai.

No. We have no audio task type, so we don’t distill speech-to-text or text-to-speech models. Where we fit in a voice pipeline is the text layer between them: intent detection, routing, tool calling and response generation, which is where much of the per-call cost and latency sits.

When your task’s input and output are stable enough that you would write an eval for it. If the task still changes week to week, stay on a frontier model. Once it settles into a repeatable pattern, that is the point to distill it.

No, we work across industries. Any narrow high-volume task qualifies. In practice the best fit is AI-native companies running LLMs in production at real volume, where one task is big enough that the cost shows up on the bill.

On the narrow task you train for, the tuned student matches the teacher it learned from, and on tight tasks it often beats it, at a fraction of the cost and latency. It will not beat a frontier model at open-ended general reasoning, and we don’t claim it does. Every training run scores the tuned student against the untrained one on your own eval set, and a separate teacher evaluation gives you the teacher’s score on that same set, so you see the comparison on your data rather than a public leaderboard. Read the benchmarks.

It depends on the student you pick. SmolLM2 is 8k at the low end, Qwen3, Gemma 3 and LFM2.5 are 32k, Llama 3.x and Gemma 4 reach 128k, and the whole Qwen3.5 family is 262k. In practice we deploy up to around 64k, because a bigger window costs more to serve: the KV cache grows with sequence length, so the usable window on constrained hardware sits well below the advertised one. Treat a large window as headroom rather than a retrieval strategy, since filling it with weakly relevant chunks makes answers worse and inference slower. See supported models.

Both, but most customers take the dedicated endpoint. We deploy the model behind an OpenAI-compatible endpoint and you pay per GPU hour of uptime. Integration is a base URL and API key change, and you never have to think about GPUs or scaling. You can also download the weights and run them yourself: that is standard under the R&D license, and commercial self-hosting needs our written consent. See pricing.

The endpoint is OpenAI-compatible, so integration is a base URL and API key swap rather than a rewrite. If you do run it yourself, distil slm download gives you the weights and distil slm download-metadata gives you a matching inference client, and you serve them with vLLM like any other open-weight model.

Use the exact prompt you trained with. The model was tuned to it, so a different prompt at inference quietly costs you accuracy: nothing errors out, you just get worse answers. Even small edits count.

What varies per request is the user message. The system prompt stays fixed. Running distil slm download-metadata gives you a client script with the trained prompt already in it.

If you need a different prompt, change it in the job description and retrain.

If your system prompt has dynamic parts, like per-customer rules or templated variables, that works too: the fixed instructions live in the job description and the model learns them, while the variable content travels in the user message on each request.

Not on the self-serve platform: those training runs happen on our infrastructure. On the Enterprise tier we deploy the whole platform inside your environment, whether that is your cloud, your VPC or air-gapped, and retraining then runs on your own hardware. If your constraint is only about where inference happens, that is simpler: self-hosting the model is standard under the R&D license, and commercial self-hosting needs our written consent. See pricing.

Yes, that is the point of a small model. For local use, distil slm download gives you the weights and you serve them yourself, which is how people run the sub-1B models on a laptop. On the server side, to give you a sense of scale, a fine-tuned Qwen3 4B served by vLLM occupied 7.6 GiB and sustained 222 requests per second on a single H100 in our inference-cost benchmark. Read the benchmark.

For quality, you keep the held-out eval set from training, so you can re-run it against production samples to check for drift. That is the part that catches real regressions. For traffic and latency, we don’t ship a metrics dashboard today: the endpoint is OpenAI-compatible, so point it at whatever you already use to instrument your LLM calls and nothing changes on your side.

You own it. It is your model, trained on your task, and because both the student and the teacher are open-weight, no third party has a license claim on what comes out. Where you can run it depends on the license: downloading and self-hosting is standard for R&D, and commercial use runs on our platform unless we give written consent to self-host. For that, contact us at contact@distillabs.ai. See pricing.

No. We don’t train on other customers’ data, and the models are single-use. Our Terms and Conditions state your traces are never used to train the base model or models for other customers, and the model is tuned to your prompt so it would not transfer anyway.

No. We delete the model, the training artifacts and your data when the agreement ends, or sooner if you ask us in writing, and we provide proof of deletion on request. See the Terms and Conditions.

Commercial use goes through our platform, so self-hosting commercially needs our written consent. You also can’t re-train the model outside the platform or use it to build a competitor. Everything else, including the acceptable use rules, is in our Terms and Conditions.

The first two training runs are free, so you can start without a card. After that, if you host with us, you get a dedicated endpoint and pay per GPU hour of uptime, not per token. The per-token figures on the pricing page are what that works out to at sustained load. See pricing.

You get both: the eval report and the model itself. You can spin up a deployment to try it, though that is a session rather than a permanent endpoint, and it stops after six hours or after an hour with no traffic. A dedicated endpoint that stays up is the paid tier. You can also download the weights and run them yourself. See pricing.

Sign up and train one, free. You get two full training runs. You need a task description plus either a few hundred of your agent traces or 20+ labeled examples, and the model comes back in under 24 hours. The quickest route is to point your coding agent at our onboarding script: give Claude Code the prompt Hey, pull the contents of distillabs.ai/onboarding.md and get me onboarded from the instructions, and it installs the CLI, creates your account and trains a first model with you, stopping for your go-ahead along the way. Then download the weights and run it locally to see how it does on your task. If you would rather do it with us, we also run a short onboarding call. Start training.