Learn
Practical guides to fine-tuning, distillation, and deploying small language models.
Run a Fine-Tuned SLM with llama.cpp
Serving a distilled small language model through llama-server, including the exact flags the distil CLI passes, why the Jinja chat template matters, and how to confirm the endpoint is serving your model and not a default one.
Running a Small Language Model on CPU
CPU inference for a distilled SLM: how to size the model to available RAM, which quantization to run, how to set thread counts, and the published tokens-per-second figures that tell you whether your workload is realistic.
Three Ways to Get Training Data for a Small Language Model
A decision guide to the three routes for getting training data onto the distil labs platform — live traffic collection, uploading existing traces, or a minimal hand-written dataset.
Train an SLM for Content Moderation
A moderation model inherits whatever your policy leaves undecided — how to convert policy into labels, seed from overturned appeals, and keep the model inside a review path it cannot bypass.
Train an SLM for Document QA over Contracts
Contract QA lives or dies on chunking: a clause split across two chunks produces a confidently wrong answer. How to segment agreements, build open-book training data, and grade the result.
Train an SLM for Intent Detection
Intent detection is decided at the boundaries between intents, not in the middle of them — how to derive intents from backend actions, mine ambiguous utterances, and handle out-of-scope input.
Train an SLM for PII Redaction
Redaction has asymmetric costs: a missed identifier is a disclosure, a spurious one is noise. How to write the policy, seed the obfuscated cases, and grade a model that has to be exhaustive.
Train an SLM for Support Ticket Triage
Ticket triage fails on taxonomy design, not on model capacity — how to write class descriptions your annotators agree on, seed the confusable pairs, and train a router with the distil CLI.
Train an SLM for Text-to-SQL
Text-to-SQL fails on schema grounding rather than on SQL syntax — how to feed the schema, seed across the difficulty range, and evaluate queries that are correct without matching the reference string.