A small language model is one you can realistically run yourself. That is a deliberately practical definition rather than a parameter count, because the threshold moves: models in the 270M to 8B range now handle tasks that needed a frontier API two years ago, provided they are trained for the specific job.
The selection question breaks into two parts that people often merge. Which student do you deploy, and which teacher generates the training data? They have opposite constraints. The student has to fit your latency, cost, and hardware budget, because you run it on every request. The teacher never gets deployed at all, so its only job is to be good at your task.
What size actually buys you
Bigger students reach a given quality bar with less training data, and they degrade more gracefully when the data is imperfect. Smaller students are faster and cheaper, and they open up deployment options — on a phone, on a CPU, inside an air-gapped network — that larger ones close off.
The rough shape, though the boundaries are soft:
| Size | Typically good for |
|---|---|
| Under 500M | Narrow classification, constrained tool calling, on-device work |
| 1B–2B | Most single-task work; the usual starting point |
| 3B–4B | Tasks with reasoning steps or long outputs |
| 8B | When smaller students have measurably fallen short |
Starting at 1.7B and moving only when the metrics justify it is the sane default. The common mistake is the opposite: reaching for the largest available student, hitting the quality bar, and never discovering that a model a quarter of the size would have done the same job.
Base model quality is a weaker signal than it looks
The intuitive assumption is that the best base model produces the best fine-tuned model. Our benchmarking across twelve small models and eight tasks found that relationship is far noisier than expected — a model’s out-of-the-box score is a poor predictor of where it lands after task-specific training. Some models that benchmark modestly turn out to be excellent students.
The practical consequence is that leaderboard position is not a shortcut. If the choice matters for your task, the cheap move is to train two candidates and compare, rather than reasoning from general benchmarks.
Architecture families
The catalog spans several families — Qwen, Llama, Gemma, SmolLM, and Liquid’s LFM series — and they are not interchangeable. Some have stronger built-in tool-calling behaviour, which gives them a head start on function-calling tasks. Others are tuned for on-device efficiency. Tool calling in particular only works with a subset of models, so it constrains the choice earlier than other task types do.
Where this cluster goes
The articles below include spotlights on individual students with their benchmark numbers and known weaknesses, head-to-head comparisons for specific task types, VRAM and sizing guidance, and a walkthrough of the base-model-accuracy finding above.
For what happens after selection, see fine-tuning and training data. For running the result, see deployment.