SmolLM2 135M: The Smallest Useful Student
SmolLM2-135M-Instruct is the smallest student in the distil labs catalog. Fine-tuned on a narrow classification or extraction task it is genuinely useful; it is also English-only, capped at an 8K context, and never appears in the top six of any of our published fine-tuning rankings.
Is 135M parameters enough to be useful?
Yes, for one job at a time. The model was trained on 2 trillion tokens according to the SmolLM2-135M-Instruct card — a very high token count per parameter, which is why it behaves better than its size suggests on tasks with a bounded output space.
“Useful” here means: after fine-tuning, it can hold a decision boundary over a fixed label set, or fill a fixed template. It does not mean it can do two of those things at once, and it does not mean it can do either without training.
What does SmolLM2 135M actually run on?
Anything. That is the entire argument for it. Weight storage falls out of the parameter count directly:
| Precision | Bytes per parameter | Weight size |
|---|---|---|
| BF16 | 2 | 135M × 2 = 0.27 GB |
| INT8 | 1 | 135M × 1 = 0.14 GB |
| 4-bit | ~0.5 | 135M × 0.5 = ~0.07 GB |
Those are computed from the parameter count, not measured. At 70 megabytes of 4-bit weights you are below the size of many mobile app bundles, which puts it in browsers, embedded controllers, and CPU-only containers where a 1B model is not an option. Fine-tuning without a GPU and running on-device are both realistic at this scale.
Where does SmolLM2 135M break down?
English only. The card states the models “primarily understand and generate content in English.” Every other family in the catalog is multilingual to some degree. If your input has a second language in it, this model is disqualified before any accuracy discussion.
8K context. That is the shortest window of any student in the catalog — Gemma 3 and LFM2.5 offer 32K, Qwen3.5 offers 262K. Document-level tasks and long RAG contexts do not fit.
No tool calling on the platform. The card mentions function calling among the model’s capabilities, but the distil labs catalog restricts tool-calling tasks to the Qwen3, Qwen3.5, Llama 3, LFM2/LFM2.5, FunctionGemma, and Gemma 4 families. SmolLM2 is not among them. If you want a tiny function caller, use FunctionGemma 270M or LFM2.5 350M.
It does not win rankings. Our 12-model benchmark found the Qwen3 family taking four of the top six fine-tuned spots with “Gemma and SmolLM2 trailing behind”, and SmolLM2-135M does not appear in the top six of the fine-tuned, tunability, or base-performance tables. The larger SmolLM2-1.7B reached fifth on tunability at 5.00; the 135M did not place.
Factual reliability. The authors recommend treating the models “as assistive tools rather than definitive sources of information.” Believe them at 135M.
How does it compare with the other sub-500M students?
| Model | Parameters | Context | Languages | Tool calling on distil labs |
|---|---|---|---|---|
| SmolLM2 135M | 135M | 8K | English | No |
| Gemma 3 270M | 270M | 32K | Multilingual | No |
| FunctionGemma 270M | 270M | 32K | Multilingual | Yes |
| LFM2.5 350M | 350M | 32K | 9 languages | Yes |
SmolLM2 135M loses on every column except size. It is half the parameters of the next model up, and that is the only reason to choose it.
When should you pick it over a 270M or 350M model?
When the size difference decides whether the thing ships at all. Concretely: a hard binary size budget, a device with no accelerator, or a per-request cost target that a 270M model misses and a 135M model hits.
If none of those apply, start higher. The cheapest way to find out is to train both — training two students at platform defaults costs less than arguing about it, and base-model accuracy does not predict fine-tuned accuracy, so desk research will not settle it. Run teacher evaluation first: if the teacher cannot solve your task, no student at this size will. Broader sizing guidance is in what size model do you need.