Build vs Buy: Training Your Own SLM
Building means owning the whole pipeline: data generation, filtering, hyperparameter search, evaluation harness, packaging. Buying means owning only the task definition and the seed examples. The deciding question is whether the pipeline itself is something your team needs to control, not whether you can operate it.
What does building actually involve?
Fine-tuning is one step out of six, and it’s the easy one. The stages below are what a working pipeline contains regardless of who runs it.
| Stage | What it involves | Where the time goes |
|---|---|---|
| Task definition | Write the spec, agree labels | Disagreement between reviewers |
| Seed data | Collect and label real examples | Sourcing, not labelling |
| Data expansion | Generate and filter synthetic examples | Building the validator |
| Training | LoRA or full fine-tune | Mostly waiting |
| Hyperparameter search | Learning rate, epochs, rank | Compute and orchestration |
| Evaluation | Held-out harness, judge prompts | Trusting the number |
The tooling for the training stage is mature and free. Axolotl covers full fine-tuning, LoRA, QLoRA, preference tuning and RL from a single YAML config; Hugging Face TRL provides trainers for SFT, DPO, GRPO and knowledge distillation. LoRA itself made the compute requirement small enough that a single GPU is often enough. Whether an adapter is enough for your task is the question in LoRA vs full fine-tuning.
None of that helps with rows two, three and six, which is where most of the effort actually sits.
When is building the right call?
Building is right when the pipeline is part of your product or your constraint set, not just a means to a model. Four cases where it’s clearly the correct decision:
You’re training many models, continuously. If you retrain dozens of task models on a schedule, the fixed cost of a pipeline amortises and per-model pricing doesn’t. Owning it is cheaper and faster to iterate.
Nothing may leave your infrastructure. If your training data can’t cross a network boundary at all (classified material, regulated health data under a strict interpretation, air-gapped environments), a hosted platform isn’t an option regardless of its terms.
You’re doing research, not production. Novel objectives, unusual architectures, custom RL rewards. Any platform is a constraint here, and constraints are the wrong shape for research.
You already have the team and the data. If you have an ML engineer and 10,000 clean labelled examples, most of the value a platform adds is value you’ve already created. Point Axolotl at it.
When is buying the right call?
Buying is right when the model is a means to an end and your bottleneck is data rather than compute. That describes most product teams.
The distil labs platform benchmarking makes the trade concrete: the seed-only baseline (a student trained on the initial labelled set with no synthetic expansion) sits well below the fully trained student on every task, and dramatically below on tool calling. Building a pipeline that doesn’t include synthetic generation and validation gets you the seed-student number.
Buying is also the right call when you need the failure to be cheap. Teacher evaluation runs a large model against your test set before any training happens, so a badly-specified task fails in minutes rather than after a week of pipeline work.
The minimal path is short enough to be the entire evaluation:
distil seed-dataset create --data ./my-data-folder
distil training-dataset create-from-seed-dataset <seed-dataset-id>
distil slm create-from-training-dataset <training-dataset-id>
distil slm download --destination ./model <slm-id>Note the last line. Buying the training doesn’t mean renting the model. The student is open-weight and you can serve it yourself, locally or on your own GPU.
What do the cost comparisons usually miss?
Three things, all of which favour whichever option the comparison was written to sell, so check them yourself against what it costs to fine-tune a small language model.
- Synthetic data generation is the expensive part, not the GPU. Generating and validating thousands of examples means many teacher-model calls plus a filtering layer you have to write. The traces benchmark shows why the filtering matters: training directly on raw production traces scored 14–28 percentage points below the curated ceiling, while the synthetic-from-traces pipeline stayed within 2 points.
- The evaluation harness is a real project. Judge prompts, held-out splits, and variance measurement are the difference between a number you trust and a number you quote.
- Maintenance never appears in the build estimate. Base models get superseded, dependencies drift, and someone has to re-run the pipeline when the task changes.
Against that, buy-side comparisons routinely understate lock-in risk and overstate how bounded your task will stay. Both distortions are worth correcting before you decide.
Which mistakes make this decision go badly?
Deciding before you have a test set. Neither path works without held-out examples you believe. Build that first; it’s the only artefact that’s valuable under both options.
Building the pipeline to learn, then shipping it. A learning exercise is a good reason to build. It’s a bad reason to put the result on the critical path of a product.
Buying to avoid the data work. No platform removes the need to define the task and supply seed examples. If you can’t describe correct behaviour in a paragraph, see when not to use a small language model before spending money either way.
Most teams land in a hybrid: buy the first model to establish that the task is trainable and worth the effort, then decide whether the second through tenth justify owning the pipeline. Is fine-tuning worth it covers the prior question, and alternatives to the OpenAI fine-tuning API covers the specific tools on the build side.