Train an SLM for Voice Assistant Command Routing

In voice, latency chooses the model. A cascaded assistant has roughly 500–800ms per turn before the conversation stops feeling natural, ASR and TTS have already spent most of it, and what remains sets the size of the model that routes commands. Train multi-turn-tool-calling-closed-book against that budget.

Design backwards from the latency budget

Conversational UX research places the threshold for natural-feeling interaction at 500–800ms end to end. Our voice assistant analysis measured the cascade against that ceiling:

Stage	Typical latency
ASR (speech to text)	~200ms
Brain — cloud LLM	500ms–1.2s depending on provider
Brain — fine-tuned Qwen3-0.6B, self-hosted	40–100ms local, ~200ms with network
TTS (text to speech)	~75ms

With a cloud brain the pipeline is over budget before TTS starts; the brain stage alone consumes over 70% of total processing time. With a locally served sub-1B student it fits with room to spare. That is the whole architectural argument, and it is a latency argument, not a cost one — though the cost follows.

Fix your budget first, subtract ASR and TTS, and only then pick a student. Everything below assumes that ordering.

Step 1: Fix the output contract

The SLM must never generate user-facing text. It emits one structured tool call per turn — intent plus slots — and a deterministic orchestrator renders every spoken response from templates.

This is not a stylistic preference. It bounds latency (no long generations), guarantees well-formed output (the schema is checked), and keeps brand voice out of the model’s hands. It also means a 0.6B model is sufficient: it never has to write, only to classify and fill slots.

Step 2: Write the tool schema for the operations you actually have

One function per backend operation, with the slots as typed parameters and enum wherever the value set is closed. The banking assistant in the reference implementation covers 14 operations — balance checks, transfers, card cancellation, fraud reporting and the rest.

{
  "type": "function",
  "function": {
    "name": "transfer_funds",
    "description": "Move money between two of the customer's own accounts",
    "parameters": {
      "type": "object",
      "properties": {
        "amount": {"type": "number", "minimum": 0.01},
        "from_account": {"type": "string", "enum": ["checking", "savings"]},
        "to_account": {"type": "string", "enum": ["checking", "savings"]}
      },
      "required": ["amount", "from_account", "to_account"]
    }
  }
}

Constrain hard. Every enum you write is a class of ASR mis-transcription the model cannot route into.

Step 3: Seed conversations with real speech artefacts

Voice input is not typed input. Seed data written by hand from a product spec produces a model that fails on the first real caller.

Include, deliberately: filler words and self-corrections (“um I need to like cancel my uh debit card”), slots arriving across turns (“I want to transfer money” → “200 dollars from checking to savings”), and mid-conversation intent changes (“actually, what’s my checking balance first?”). About 50 example conversations covering the workflow is enough to start.

Each JSONL line holds the whole conversation in one messages array, ending with the assistant call to be learned. The multi-turn data preparation guide has the format; note that arguments is a real JSON object and each assistant turn carries exactly one call.

Step 4: Train with a tool-calling-capable teacher

distil model create voice-command-router
# Output: Model created with ID: <model-id>

distil model upload-data <model-id> --data ./data
distil model run-teacher-evaluation <model-id>
distil model teacher-evaluation <model-id>

distil model run-training <model-id>
distil model training <model-id>

base:
  task: multi-turn-tool-calling-closed-book
  student_model_name: Qwen3-0.6B
  teacher_model_name: zai.glm-5
synthgen:
  teacher_temperature: 0.6

Both constraints from the supported models catalog bite here. The student must come from the Qwen3, Qwen3.5, Llama 3, LFM2/LFM2.5, FunctionGemma, or Gemma 4 families — Qwen3-0.6B qualifies, Gemma 3 does not. The teacher must be ticked for tool calling; zai.glm-5 is, and as a reasoning teacher it needs teacher_temperature in 0.5–0.7.

Step 5: Measure the brain stage in isolation

Score accuracy with tool_call_equivalence, then measure latency separately with ASR and TTS removed. A blended end-to-end number hides which stage regressed.

Accuracy compounds across turns, so a per-call figure understates the risk: at 90.9% per turn, a three-turn conversation completes correctly about 75% of the time. On the banking taxonomy a fine-tuned Qwen3-0.6B reached 90.9% against its 120B teacher’s 87.5%, with the untuned base model at 48.7% — those runs used GPT-oss-120B as teacher. Two other students have been measured on the same banking task: LFM2.5 350M at 95.9% and FunctionGemma 270M at 90.86%. See multi-turn tool calling explained for why the multiplication matters.

Step 6: Deploy behind a deterministic orchestrator

distil model deploy local <model-id>
distil model invoke <model-id>

The orchestrator owns everything the model does not: checking whether required slots are filled, asking a templated clarifying question when they are not, executing the call, and rendering the response. That division is what makes a sub-1B model production-viable — the residual errors land as “could you repeat the amount?” rather than as a wrong transfer. The VoiceTeller reference implementation is a working example of the pattern.

When a voice SLM is not the answer

When the assistant has to hold open-ended conversation. Everything above depends on a bounded set of operations with typed slots; open-ended generation still favours larger models, and no amount of fine-tuning changes that.

Also reconsider if your operations are still changing weekly — each change is a retrain. And if some turns are genuinely hard, a cascade that defers to a larger model keeps the small model on the fast path. For student sizing under a hardware ceiling, see which SLM fits in 4GB of VRAM.