What Is Catastrophic Forgetting?
Catastrophic forgetting is the loss of previously learned capability when a model is trained on something new. In practice it shows up as a fine-tuned model that scores well on its target task while general abilities it had before training — other formats, other languages, ordinary conversation — degrade or disappear.
What does catastrophic forgetting look like?
It looks like a narrow win and a broad regression, discovered late. Your task metrics improve exactly as intended, then someone sends the model an off-task prompt and gets a malformed answer, a refusal, or the trained output schema applied to a question that had nothing to do with it.
Typical reports:
- The model answers every prompt in the trained output format, including prompts that call for prose.
- Instruction following outside the task collapses: it ignores “answer in one sentence” or system-prompt constraints it previously respected.
- Multilingual ability drops even though the training data was monolingual.
- The model has become confidently wrong on general knowledge questions the base checkpoint handled.
The key diagnostic is that the trained task is fine. If the trained task is also worse, you have a different problem — see why did my fine-tuned model get worse.
What causes it, in order of likelihood?
Ranked by how often each one is the real explanation:
| Rank | Cause | Distinguishing signal |
|---|---|---|
| 1 | Deliberate task narrowing | Off-task behaviour is gone, on-task behaviour is excellent, and nobody ever asked for the off-task behaviour |
| 2 | Too many epochs over a narrow dataset | Even near-task variants degrade, not just distant ones |
| 3 | Full fine-tuning instead of adapters | The base weights themselves changed; nothing can be reverted at serving time |
| 4 | Training data covers one slice of the input space | The model handles seed-set phrasing and nothing else |
| 5 | Sequential fine-tunes stacked on one checkpoint | Each new task erases the previous one |
Cause 5 is the one the research literature measures directly. Luo et al. studied catastrophic forgetting during continual instruction tuning across models from 1B to 7B parameters and observed forgetting generally across the sizes tested, with severity increasing as models got larger. They also found decoder-only models retained more than encoder-decoder models, and that prior general instruction tuning reduced subsequent forgetting.
How do you fix each cause?
Match the fix to the cause rather than reaching for the same lever every time.
| Cause | Fix |
|---|---|
| Deliberate narrowing | Nothing to fix in the model. Route off-task traffic to a general model instead. |
| Too many epochs | Lower num_train_epochs in the config file; the platform default is 4. |
| Full fine-tuning | Keep use_lora: true so base weights stay frozen and the adaptation stays a separable artefact. |
| Narrow data | Broaden generation — mutation_topics and basic_mutators_to_use exist to spread the synthetic set across phrasings and complexity levels. |
| Stacked fine-tunes | Train one adapter per task from the same base checkpoint rather than chaining runs. |
The adapter fix is structural, not a hyperparameter. LoRA freezes the pretrained weights and injects trainable rank-decomposition matrices, so the original model is still intact on disk after training. If a fine-tune turns out to have narrowed the model too far, you detach the adapter and you are back where you started — an option full fine-tuning does not leave you.
Is forgetting always a problem?
No, and treating it as one is the most common mistake here. When you distil a task-specific small model, you are deliberately trading general capability for reliability on one job.
That trade is the point of the approach. As the write-up on the intelligent harness puts it, fine-tuning narrows the model to exactly the output format the system needs rather than preserving general-purpose capability it will never use — and in that case the resulting 0.6B student outperformed its 120B teacher on the target task.
So the question is not “did the model forget?” but “did it forget something a caller will actually ask for?” If your model sits behind a classifier endpoint that only ever receives support tickets, its lost ability to write sonnets costs nothing.
How do you prevent it?
Prevention is mostly a scoping decision made before training, not a recovery step afterwards.
- Pick a task type and stay inside it. The task selection guide exists because a well-bounded task is what makes a small student viable in the first place.
- Write down the capabilities you need to keep. Then evaluate them explicitly after training, alongside your task metrics. Anything you do not test, you will not notice losing.
- Keep the adapter separable. A frozen base plus a detachable adapter makes forgetting reversible.
- Do not stack tasks onto one checkpoint. Fresh adapter, same base, per task.
- Check against the base model, not only the teacher. The model training guide frames success as landing close to the teacher on your task; add a base-model comparison on your retained capabilities to catch the other half.
If you are still deciding between a frozen base and a full weight update, LoRA vs full fine-tuning covers the wider trade-off.