Our research paper on generating multi-turn tool-calling training data for small language models has been accepted to the SLM-Agents workshop at NeurIPS 2026. The paper introduces Turnslide, a method that grew out of Aaron Fainman’s internship at distil labs, and we will present it at the workshop in Paris on 13 December.
Abstract
Small language models are inexpensive to serve and can run on private infrastructure, but base models are often not good enough at multi-turn tool calling, and fine-tuning them needs per-API data that rarely exists. Existing synthesis methods are too expensive for high-scale fine-tuning, as they often require mock operational environments for different domains and multiple LLM calls per generated conversation turn. We introduce a fully automated, lightweight synthesis framework that models each API as a finite-state machine, representing the system as abstract states that determine when each tool may be called, producing state-valid sequences of tools; sequences are translated into complete examples with a single LLM call. Rather than optimize diversity, we set a target distribution over the number of turns, the tool sequence and task complexity. We measure data quality by fine-tuning SLMs on generated trajectories, showing that our FSM-based generation significantly improves downstream accuracy over an unmutated baseline and, against existing works, reaches 70.7% full accuracy over 63.4% and 53.7% with 3.6–6.6× fewer tokens.
Read the full paper
Turnslide: Scalable Multi-Turn Data Synthesis by Walking a Finite-State Machine
Aaron Fainman (Imperial College London), Gabriela Kadlecová (distil labs), Maciej Gryka (distil labs), Bartosz Kruszczyński (distil labs), Usman Zafar (distil labs), Cédric Archambeau (Agon), Aaron Klein (ELLIS Institute Tübingen), David Salinas (ELLIS Institute Tübingen), Selim Nowicki (distil labs), Jacek Golebiowski (distil labs).
Download the paper (PDF) or read it on arXiv.