FunctionGemma 270M for Multi-Turn Tool Calling
Out of the box, FunctionGemma 270M gets between 9.9% and 38.8% of multi-turn tool calls exactly right. After fine-tuning on distil labs it reaches 90.9% to 96.7%, matching or exceeding a 120B teacher. The model is a starting point, not a product.
Can FunctionGemma 270M do multi-turn tool calling out of the box?
No, and Google says so on the card. The FunctionGemma 270M IT model card states the model “is intended to be fine-tuned for your specific function-calling task, including multi-turn use cases” and is “not intended for use as a direct dialogue model.”
Our FunctionGemma benchmark measured what that means, in tool call equivalence:
| Task | Base | 2-turn (base²) | 5-turn (base⁵) |
|---|---|---|---|
| Smart home control | 38.82% | ~15.1% | ~0.9% |
| Banking voice assistant | 23.35% | ~5.5% | ~0.07% |
| Shell command execution (Gorilla) | 9.90% | ~1.0% | ~0.001% |
Why do small errors compound across turns?
Because a conversation succeeds only if every turn succeeds, so accuracies multiply. That arithmetic is unforgiving in both directions.
At 80% per-call accuracy you finish a five-turn conversation correctly about 33% of the time (0.8⁵). At 95% you reach 77%. The distance between “usually right” and “reliably right” is much larger than the single-turn numbers suggest, which is why single-turn benchmarks mislead people who are building multi-turn agents.
The observed failure modes are structural rather than subtle: free text where JSON was required, hallucinated function names outside the catalogue, slot values dropped between turns, and malformed output that breaks the parser downstream.
What does fine-tuning change?
Everything that matters, on the same three tasks and the same evaluation set.
| Task | Teacher (120B) | Base | Tuned |
|---|---|---|---|
| Smart home control | 92.11% | 38.82% | 96.71% |
| Shell command execution | 97.03% | 9.90% | 96.04% |
| Banking voice assistant | 96.95% | 23.35% | 90.86% |
Shell commands moved almost 10x, from 9.90% to 96.04%. Note also that the same curated dataset produced a strong Qwen3-0.6B model with no model-specific adjustments — the data is doing the work, not per-model tuning. This is knowledge distillation doing what it is supposed to do: a 270M student learning the behaviour of a model 445 times its size on one bounded task.
Where does FunctionGemma still lose to the teacher?
On the banking voice assistant, by about 6 points. That task has 14 banking operations with varied slot types, ASR transcription artefacts in the input, and users who change intent mid-conversation. It is the hardest of the three and it stays hardest after training.
90.86% is a viable production number with an orchestrator. The slot-elicitation loop — asking a clarifying question when confidence is low — catches most of the residual. It is not a viable number for a system that executes calls without a confirmation path.
Compare with LFM2.5 350M on the same three tasks: it starts 2-6x higher and finishes ahead on the voice assistant at 95.9%. If you can afford 350M instead of 270M, that is the stronger starting point.
What should you not use FunctionGemma for?
Anything that is not function calling. It is not a dialogue model, not a summarizer, and not a classifier — the card is explicit, and its training reflects that specialisation.
Two more limits worth naming. First, the card lists a 32K total context for input and output combined, which caps how much conversation history you can carry. Second, it inherits the general Gemma limitations: it can “generate incorrect or outdated factual statements” and struggles with “subtle nuances, sarcasm, or figurative language” — all irrelevant for schema-filling, all disqualifying for open text.
What it is genuinely good for is a small, offline, structured dispatcher. At roughly 288MB quantized and around 125 tokens per second on a phone CPU, it fits where a multi-billion-parameter model cannot go. Start from the multi-turn tool calling data preparation guide, write 20-100 seed conversations, and let the teacher expand them — few-shot fine-tuning from 10 examples explains why that is enough to start.