Learn
Practical guides to fine-tuning, distillation, and deploying small language models.
The 10x Inference Tax
The inference tax is the premium you pay for running a general-purpose frontier model on a task a specialised small model handles equally well. Here's where the multiple comes from.
Train a Model from 1% of Production Traffic
The lowest-effort route to a task-specific small model: let a small share of your live traffic be collected as training material, with nothing for you to write or label.
What Is a Held-Out Test Set?
A held-out test set is data the model never saw during training or synthetic data generation. In a distillation pipeline that second clause is where contamination usually gets in.
What Is a Small Language Model?
A small language model is a language model small enough to fine-tune and serve on commodity hardware, roughly 100M to 10B parameters, and the size band where task-specific training pays off most.
What Is a Student Model in Knowledge Distillation?
The student model is the small model you fine-tune on teacher-generated data and actually deploy. Sizes, how it learns, and what it gives up compared with the teacher.
What Is a Teacher Model in Knowledge Distillation?
A teacher model is the large LLM that generates and validates synthetic training data for a smaller student. Here's what it does, which models qualify, and why you never deploy it.
What Is GGUF?
GGUF is the single-file binary format that packages a model's tensors and its metadata together for llama.cpp-family runtimes: what it stores, what its quantization type names mean in bits per weight, and when it's the wrong choice.
What Is LLM-as-a-Judge Evaluation?
LLM-as-a-judge scores model output by asking a large model to grade it: the default metric for generation tasks on distil labs, and what it costs you to trust one model's opinion of another.
What Is LoRA Rank?
LoRA rank is the inner dimension of the low-rank update matrices injected into a frozen model, the one knob that sets how much an adapter can learn.