Learn
Practical guides to fine-tuning, distillation, and deploying small language models.
What Is Tool Calling and How Do You Train for It?
Tool calling turns a natural-language request into one schema-valid function call. What the task type expects in your files, which models are eligible, and the measured jump fine-tuning produces on sub-1B students.
What Makes a Good Production Trace?
What the trace processing pipeline scores a conversation on — relevance, coherence, and repairability — and why a trace can be a poor training label yet still be excellent seed data.
Why Every Path to Training Data Ends in Synthetic Data
However you supply seed data — live traffic, uploaded traces, or a hand-written set — the platform converts it into synthetic training data before any fine-tuning happens. Here is why.
Knowledge Distillation Explained: Teacher-Student Training for LLMs
Learn how knowledge distillation works — the teacher-student training process that compresses large language models into small, fast, deployable models without losing accuracy.
Knowledge Distillation for LLMs: Compress GPT-4 into a 3B Model
Learn how knowledge distillation lets you compress the capabilities of massive language models like GPT-4 and Llama 70B into small, deployable models with 1B–8B parameters — without sacrificing accuracy on your task.
Teacher-Student Distillation: How It Works and When to Use It
Learn how teacher-student distillation transfers knowledge from a large language model to a small, efficient one. Understand the training process, when it makes sense, and how to get started.
What Is Catastrophic Forgetting?
A fine-tuned model that nails its new task but has lost the general abilities it arrived with has forgotten catastrophically — here is how to spot it, fix it, and when to accept it.
What Is Overfitting in Fine-Tuning?
Overfitting is when a fine-tuned model memorises its training examples instead of the pattern behind them — spot it from the held-out gap and fix it with data before hyperparameters.
When Does Distillation Fail?
Distillation fails on under-specified tasks, inconsistent labels and open-ended outputs. Ranked causes, the fix for each, and the teacher evaluation step that catches most of them early.