Learn
Practical guides to fine-tuning, distillation, and deploying small language models.
Deploy an SLM with Ollama
Loading a fine-tuned small language model into Ollama from a local GGUF or straight from Hugging Face, writing a Modelfile that preserves the training-time system prompt, and the chat-template trap specific to this runtime.
Deploy an SLM with vLLM
Serving a distilled small language model on a GPU with vLLM: environment setup, the tool-calling parser flags, querying the OpenAI-compatible endpoint, and the throughput and memory figures we measured on a single H100.
Do You Actually Need a Bigger Model?
Sometimes yes: broad reasoning and free-form generation still favour scale. But on bounded tasks the published rankings invert after fine-tuning, and this guide separates the two cases.
Fine-Tune, RAG, or Prompt: Which Should You Use?
Three techniques that fix three different problems: missing facts, wrong behaviour, and an under-specified task. A criteria table for telling them apart before you commit engineering time.
Fine-Tune a Model With 20 Examples
The minimal-dataset route end to end: four files, twenty labelled examples, and the distil CLI commands that turn them into a trained small language model.
How to Choose a Teacher Model for Distillation
Choose a teacher on measured accuracy for your own task, not parameter count. Criteria table, tool-calling constraints, and the teacher evaluation step that settles it.
How to Deploy a Fine-Tuned Small Language Model
The four ways to put a trained SLM into service (managed endpoint, local llama.cpp, self-hosted vLLM, and an offline download), with the constraint that selects each one and the mistakes that quietly break accuracy.
Run a Fine-Tuned SLM with llama.cpp
Serving a distilled small language model through llama-server, including the exact flags the distil CLI passes, why the Jinja chat template matters, and how to confirm the endpoint is serving your model and not a default one.
Running a Small Language Model on CPU
CPU inference for a distilled SLM: how to size the model to available RAM, which quantization to run, how to set thread counts, and the published tokens-per-second figures that tell you whether your workload is realistic.