Training a task-specific model is not always the right move, and a page arguing otherwise would not be worth reading. This cluster covers the honest comparisons — including the cases where you should do something else.
The three-way choice
Most teams are really choosing between prompting a frontier model, retrieval, and training a small one. They solve different problems, and the confusion comes from the fact that all three can produce a working demo.
Prompting changes the instructions. It is the fastest to iterate on and requires no infrastructure, which is why almost everything starts here. It stops being the right answer when per-request cost becomes material, when latency matters, when outputs need to be consistent rather than prompt-sensitive, or when the data cannot leave your network.
Retrieval changes what the model sees, by fetching relevant context at query time. It is the right answer when the knowledge changes frequently, or when you need to show which source justified an answer. A fine-tuned model cannot point at a document; retrieval can.
Training changes the weights. It is the right answer for a well-defined, high-volume task where you want a small, fast, self-contained model with predictable behaviour.
These are not mutually exclusive. Fine-tuning a small model to handle retrieved context well is a common and effective combination.
When not to train
Worth stating directly:
- The task is not well defined. If you cannot describe what a correct output looks like, you cannot generate training data for it or measure whether it worked.
- Requirements change weekly. Retraining has a cycle time. If the specification moves faster than that, prompting is more honest.
- Volume is low. The economics only work at sustained throughput. A few thousand requests a month does not justify the operational overhead.
- You need broad general capability. A task-specific model is task-specific. If users can ask anything, you want a general model.
- No teacher can solve it. If the frontier models fail at your task, distillation has nothing to transfer.
Routing, agents, and the “smaller pieces” question
Two adjacent patterns come up often. One is routing — sending easy requests to a small model and hard ones to a large one. The other is agent frameworks that orchestrate a general model across many tools.
Both are reasonable, and both are sometimes solving a problem that a single well-trained model removes. A small model can be trained to know when to defer, which collapses the router into the model itself. A model trained on your specific tool schema can be more reliable at calling them than a general model reading tool descriptions at runtime.
Where this cluster goes
The articles below cover fine-tuning against RAG and prompting directly, combining retrieval with training, alternatives to hosted fine-tuning APIs, open versus proprietary models, build versus buy, and when a small language model is the wrong choice.