How Many Examples Do You Actually Need to Fine-Tune a Model?
Twenty. That is the documented minimum for train.jsonl, and 20 to 100 is the range that usually suffices. The number is small because your examples are seeds, not the training set — a teacher generates thousands more from them, targeting 10,000 by default.
What do the published runs show?
Two runs bracket the range: one starting from 50 hand-written examples, one from five production traces.
| Run | Seeds | Student | Result |
|---|---|---|---|
| PII redaction | 50 examples, split train/test | Llama-3.2-3B | 0.87 trained vs 0.85 teacher |
| PII redaction, seed data only | Same 50 | Llama-3.2-3B | 0.73 |
| PII redaction, no training | — | Llama-3.2-3B | 0.54 |
| Restaurant booking, low-data scenario | 5 traces | Qwen3-1.7B | 0.852 vs 0.649 direct |
The PII rows come from small expert agents from 10 examples; the trace row from our traces benchmark. Metric in both cases is LLM-as-a-judge.
Read the first three rows together, because the comparison inside them is the whole argument. Fifty seed examples took an untrained 3B student from 0.54 to 0.73 when used directly as training data. The same fifty, used as seeds for synthetic generation, took it to 0.87 — past the 70B teacher’s 0.85. The seeds did not get better. What changed was how many examples the student saw.
How was the PII result measured?
Fifty seed examples, balanced across policy cases, split between train and test. The teacher was Llama-3.3-70B evaluated with k-shot examples drawn only from the training split; the student was Llama-3.2-3B. Scoring compared predicted JSON against the reference answer with an LLM judge, reported as mean and standard deviation.
| Configuration | Score |
|---|---|
| Teacher (Llama-3.3-70B, k-shot) | 0.85 ± 0.01 |
| Trained student (seed + synthetic) | 0.87 ± 0.01 |
| Seed student (seed data only) | 0.73 ± 0.01 |
| Base student (untrained, k-shot) | 0.54 ± 0.03 |
A student beating its teacher looks wrong until you notice where the extra signal comes from: the validators. Generated examples that fail schema, length, or similarity checks never reach training, so the student trains on a filtered version of the teacher’s output rather than on the teacher’s raw behaviour. This is a departure from classical knowledge distillation, where the student can at best approach the teacher. Can a small model beat its teacher covers it properly.
Why is twenty the floor rather than two?
Because below roughly that point the seeds stop describing the task. Twenty is also enforced from the other end: min_generated_examples defaults to 20 and trace processing raises an error rather than training on fewer, a guard documented in the config reference.
The deeper reason is coverage. Generation samples from your seeds to write new examples, and it cannot invent a category of input it has never seen. Twenty examples that each demonstrate something different is a usable specification of a task. Five that all demonstrate the same thing is not, however clean they are.
This is the finding LIMA reported from the other direction: 1,000 carefully curated prompts outperformed far larger but noisier instruction sets, because what alignment training needs is coverage and consistency rather than volume.
What pushes the number up?
Four things, roughly in order of how much they move it.
| Factor | Effect | Why |
|---|---|---|
| Number of classes or tools | Large | Every class and every tool needs its own examples |
| Multi-turn interaction | Large | More patterns to demonstrate than single-turn |
| Ambiguous boundaries | Medium | Edge cases have to be shown, not described |
| Output format complexity | Medium | Nested JSON needs more demonstrations than a label |
A binary classifier sits at the bottom of the range. A multi-turn tool-calling agent with a dozen functions sits at the top and may exceed it — the platform’s own guidance is that harder problems may need more, though still orders of magnitude less than conventional fine-tuning. See what size model do you need for the parallel question on the model side.
What does not push the number up?
Volume of similar examples. The default validation_similarity_threshold is 0.95, and generated examples closer than that to your seed data are removed — so near-duplicate seeds contribute near-nothing.
This has a practical consequence that catches teams with large datasets. If you have 5,000 examples, do not upload all of them expecting a better model. Sample the most varied few dozen. The rest are more useful as unstructured.jsonl context, where they steer generation toward your domain’s vocabulary without competing as labels.
How do you know when you have enough?
Run teacher evaluation and read the number. It is the cheapest signal available and it comes before training.
A teacher that scores well means your task is well-specified and your examples describe it adequately. A teacher that scores badly means one of the two is wrong — and in our experience it is usually the job description rather than the example count. Adding examples to fix a specification problem does not work.
If the teacher is fine and the trained student is not, then look at seed coverage: find the inputs it fails on, and check whether anything like them appears in your seeds. Practical next steps are in fine-tune with 20 examples, few-shot fine-tuning with 10 examples, and three ways to get training data.