What Is Text Classification with a Small Language Model?
Text classification is the task type where a model reads an input and returns exactly one label from a fixed, known set. On distil labs you declare task: classification, supply a description of every class, and the trained model emits the label string and nothing else — no explanation, no hedging.
How does classification work on a fine-tuned small model?
The model is still generating text; the task type constrains what that text is allowed to be. A classifier trained this way outputs the label token sequence directly, which is why downstream code can consume it without a parser.
Three things make it work. The classes_description map in job_description.json tells the teacher what each class means, so it can generate synthetic examples that respect the boundaries. Fine-tuning then teaches the student to commit to a single label instead of describing what it sees. And because the output space is closed, evaluation is unambiguous — a prediction is right or wrong, with no scoring model in the middle.
Base models are bad at this out of the box, and predictably so. The failure modes are consistent: confusing overlapping categories, missing weak signals in short inputs, and producing an extra sentence around the label.
What does the training data look like?
Four files, of which two carry the task definition. The classification data preparation guide has the full spec.
| File | Required | What it holds |
|---|---|---|
job_description.json |
yes | task_description plus a classes_description map from label to definition |
train.jsonl |
yes | 20+ labelled examples, each a messages pair |
test.jsonl |
yes | Held-out examples, same shape |
config.yaml |
yes | base.task: classification and model choices |
unstructured.jsonl |
no | Unlabelled in-domain text to steer synthetic generation |
A training line is a two-turn conversation whose assistant turn is the bare label:
{"messages": [{"role": "user", "content": "Why is there a fee for getting cash?"}, {"role": "assistant", "content": "cash_withdrawal_charge"}]}The label strings in train.jsonl must match the keys of classes_description exactly. The class descriptions are worth more effort than the examples — they are what the teacher reads when it fabricates thousands of new examples per class, and a vague description produces vague synthetic data.
How well do small models classify after fine-tuning?
Well enough to match a teacher two to four hundred times their size, on published distil labs work.
| Task | Student | Base | Fine-tuned | Teacher |
|---|---|---|---|---|
| Email labelling, 10 classes | Qwen3-0.6B | 38% | 93% | 93% (GPT-OSS-120B) |
| AI-generated text detection, 2 classes | Gemma 3 270M | ~40% | 100% | 100% (GPT OSS 120B) |
Both studies used openai.gpt-oss-120b as the teacher, which is what those runs actually set; for new work the current pick is zai.glm-5. The email classifier started from 154 seed examples expanded to 10,000 synthetic ones; the slop detector started from about 50. The pattern in both is the same: the base model is near chance, and the entire gap is closed by fine-tuning, not by model size.
Note the second row’s caveat honestly — 100% is on a small held-out set from one Kaggle dataset, and the same model dropped to roughly 95% after 4-bit quantisation and to 88% on formal human emails in real-world spot checks. A number that clean is a statement about the test set as much as the model.
When is classification the wrong frame?
When the label set is not actually closed, or when one input can legitimately carry two labels.
- Open-ended output. If what you need is a sentence rather than a category, use question answering instead.
- Multi-label problems. The task assigns one category from a fixed set. Overlapping labels need either a merged taxonomy or one binary classifier per label.
- Arguments, not just a name. If the downstream system needs parameters — an account number, a date — you want tool calling, where the output is a schema-checked call.
- A taxonomy nobody can apply consistently. If two annotators disagree on 20% of your examples, the model will inherit that disagreement. Fix the definitions first.
Related terms
Class description is the per-label definition in job_description.json; it doubles as the prompt the teacher uses during generation. Seed data is your handful of hand-labelled examples before synthetic expansion — see few-shot fine-tuning from 10 examples. Unstructured data is unlabelled in-domain text that widens the distribution the teacher generates from. For choosing a student, Qwen3 1.7B vs Gemma 3 1B for classification compares the two most common candidates, and what size model do you need covers the size question generally.