Train an SLM for Intent Detection

Intent detection is a classification task where the difficulty is concentrated in a handful of adjacent intent pairs. A model that gets the clear cases right and the boundaries wrong is not usable, so build the whole recipe around the boundaries.

Intents blur where your backend does not

An intent taxonomy that reads cleanly on a whiteboard usually contains three or four pairs that no human can separate reliably. “Card not working” and “Card declined”. “Change my address” and “Update my details”. Users do not phrase requests along your product’s seams.

The fix is not more examples. It is deciding, once and in writing, which side of each boundary an ambiguous utterance falls on, and encoding that decision where both the teacher and your annotators can see it.

Scale matters here too. Intent sets are typically large — BANKING77, a standard benchmark for this task, has 77 intents in a single banking domain. At that width, adjacent-pair confusion is the dominant error mode and overall accuracy hides it.

Step 1: Derive intents from actions, not from utterances

Write down what your system can actually do: the API calls, the workflows, the handoffs. One intent per distinct downstream action. If two candidate intents trigger the same action with the same parameters, they are one intent, however differently users phrase them.

This inverts the usual process, which starts from a cluster analysis of past messages and produces intents nobody’s backend can serve. Starting from actions gives you a taxonomy that is mutually exclusive by construction, because two actions are either the same call or they are not.

Step 2: Mine the boundary utterances from real logs

Pull real user messages, not invented ones. Then sort them into three piles per intent: unambiguous, ambiguous, and out of scope. The ambiguous pile is your seed set’s centre of gravity.

Pile	How many seeds	Why
Unambiguous	5–10 per intent	Establishes the intent’s core meaning
Ambiguous between two intents	3–5 per boundary pair	Teaches the model where the line is
Out of scope	20+ total	Prevents confident misrouting of nonsense

Twenty or so examples per intent is enough for the platform to expand from — the classification data preparation guide sets that expectation — but the composition matters far more than the total.

Step 3: Encode the boundary in the class descriptions

classes_description is read by the teacher during synthetic generation, so an adjudication rule written there propagates into thousands of generated examples.

{
  "task_description": "Classify the customer utterance into exactly one intent.",
  "classes_description": {
    "card_payment_declined": "A specific payment or purchase was refused. The customer references a transaction that failed. Use this even if they also say the card 'isn't working'.",
    "card_not_working": "The card fails generally — not read at terminals, physically damaged, not activated. No specific declined transaction is referenced.",
    "out_of_scope": "The utterance is not a request this assistant can serve: chit-chat, unrelated products, or unintelligible input."
  }
}

“Use this even if they also say the card ‘isn’t working’” is the adjudication rule. Every boundary pair deserves one sentence like it.

Step 4: Upload, evaluate the teacher, train

distil model create intent-detection
# Output: Model created with ID: <model-id>

distil model upload-data <model-id> --data ./data
distil model run-teacher-evaluation <model-id>
distil model teacher-evaluation <model-id>

distil model run-training <model-id>
distil model training <model-id>

Your config.yaml:

base:
  task: classification
  student_model_name: Qwen3-1.7B
  teacher_model_name: zai.glm-5
evaluation:
  num_few_shot_examples: 1
synthgen:
  teacher_temperature: 0.6
  num_negative_exemplars_per_generation: 4

num_negative_exemplars_per_generation shows the teacher examples from other classes while it generates for the current one. Raising it above the default of 2 is the single most useful knob for a wide intent set, because it makes the teacher generate examples that are deliberately distinguishable from their neighbours. The config reference lists the rest.

Step 5: Read the confusion pairs, not the average

Download the predictions and build a swap matrix.

distil model download-training-predictions <model-id>

Expect the errors to be concentrated. On our 12-model benchmark, a fine-tuned Qwen3-4B reached 0.89 on Banking77 against a teacher’s 0.92 — the one benchmark of eight where the student fell short, and within margin of error. Compare that with TREC on the same run: 0.51 base, 0.93 tuned. A wide, semantically adjacent intent set is genuinely the hard end of classification, and the base model already scoring 0.87 on Banking77 tells you the remaining errors live somewhere fine-tuning has to work for.

When a pair keeps swapping, you have three options in order of cost: sharpen the adjudication sentence, merge the two intents, or accept the confusion and disambiguate downstream with a clarifying question.

Step 6: Ship with an out-of-scope class and a deploy check

distil model deploy local <model-id>
distil model invoke <model-id>

Test the out-of-scope class deliberately before you route anything. A classifier with no escape hatch assigns your closest intent to every input, including “what’s the weather”, and confident misrouting is worse than an honest fallback.

When intent detection should be tool calling instead

When the downstream action needs arguments. “Transfer £200 from checking to savings” carries an amount and two accounts; a label alone throws them away and you end up writing a second extraction step. Use tool calling, where the function name is the intent and the arguments are the slots, and see voice assistant command routing for that shape end to end. If your intents genuinely carry no parameters, classification stays the cheaper and more measurable choice — what is text classification covers the task type.