When a small model needs to reason, and how long it should take

TL;DR

• You can now train small models that reason on distil labs. One flag, enable_thinking: true, trains a student that writes a short reasoning block before it answers, and the style and length of that reasoning are set in synthetic_data_generation_instructions.
• Fine-tuning a small model to answer in one pass plateaus on multi-step tasks: Qwen3.5-4B reaches 84 ± 3.7 on invoice checks and 45 ± 5.0 on flight connection checks, with more than 4,000 training examples each.
• Trained on the same data with reasoning on, it scores 98 ± 1.4 and 95 ± 2.2, around its teachers (97 and 100), which are 186x and 700x its size.
• Untuned Qwen3.5-4B with thinking on does not finish inside a 2,048-token budget on 63% of invoices and 100% of itineraries, which means long generation times and no answer. The trained model reasons in a median of 272 and 208 tokens.

Why you need reasoning in SLMs

Most of the tasks we train small models for can be answered by reading the input carefully: classify this ticket, extract these fields, route this request. A model that answers in one pass does those well, and adding a reasoning step would only make each request slower and more expensive.

Some decisions inside a product are different, because the answer is not written anywhere in the input. It has to be computed from it: a total that has to be added up and compared, a time that has to be converted into another time zone before you can subtract it, a list of items that each have to be checked against a rule, in order. A model that answers in one pass has to do all of that in the single step where it writes the answer, and small models are bad at it. Reasoning gives the model room to write the intermediate values down first. Here are some examples of tasks that follow this logic:

Invoice pay or hold. This is step 2 of the accounts payable pipeline from our Jev post. The model reads a vendor invoice, the purchase order and the goods receipt, runs four checks in order (PO number, quantities, prices within 2% of the PO, the stated total) and answers with one of five decisions.

Flight connection check. A travel desk checks an itinerary before ticketing: every connection needs at least the airport’s minimum connection time (MCT) and at most 12 hours. Each leg gives only its local departure time and its flight duration, and the airport list gives each airport’s UTC offset:

Leg | Flight | From | To  | Departs (local)  | Duration
1   | QX 220 | JFK  | CDG | 2026-10-19 11:05 | 7h 40m
2   | NA 932 | CDG  | ORD | 2026-10-20 01:45 | 8h 45m
...
JFK (New York JFK): UTC-4, MCT 90 min
CDG (Paris Charles de Gaulle): UTC+2, MCT 90 min

The first flight lands at 18:45 New York time, which is 00:45 the next day in Paris, so the connection to the 01:45 departure is 60 minutes against an MCT of 90, and the answer is {"valid": false, "problem": "short_connection", "leg": 2}. If you subtract the local times without the time-zone change you get seven hours and pass it. We built every test itinerary so that this shortcut gives the wrong answer.

What happens without reasoning

To see how far a small model gets without reasoning, we fine-tuned Qwen3.5-4B to write the answer directly, with no reasoning step. It gets 84 ± 3.7 of the 100 test invoices right and 45 ± 5.0 of the 100 test itineraries. The invoice model catches every wrong total, but it also holds 14 of the 32 invoices that should be paid. Without adding up the lines it cannot confirm that a total is right, so it learned to hold when it is unsure.

The flight model does worse: its 55 errors are spread across every label and every leg, so it never learned the computation. Even a much larger model has the same problem on this task: Kimi K3 scores 17 ± 3.8 with thinking off and 100 with thinking on, with the same prompt.

How to fix this (enable reasoning on distil labs)

Both failures have the same cause: the model has to produce the answer without first writing down the values the answer depends on, such as the sum of the lines or the arrival time in Paris. So the fix is to train it on examples that include that working, in a short, fixed format, and to let it write the working before every answer. On distil labs, where a large teacher model generates the training data from a few dozen of your examples and a small student is fine-tuned on it, that takes three changes.

1. Switch reasoning on. Set one flag in config.yaml. The student is then trained, evaluated and served with a reasoning block before its answer:

base:
  task: question-answering
  student_model_name: Qwen3.5-4B
  teacher_model_name: moonshotai.kimi-k3-max-thinking
  enable_thinking: true

2. Show the working you want in a few seed examples. Add a reasoning_content field to the answer of your seed examples. For the flight task, that is one line per leg converting the arrival into local time, and one line per connection comparing the gap with the limits:

{"messages": [
  {"role": "user", "content": "ITINERARY (times local to departure airport)\n1 MF980 HNDFRA 22OCT 0950 DUR 1200\n2 NA560 FRASFO 23OCT 0725 DUR 1145\n..."},
  {"role": "assistant",
   "content": "{\"valid\": false, \"problem\": \"long_layover\", \"leg\": 2}",
   "reasoning_content": "L1 HND dep 22 Oct 09:50 UTC+9 + 12h 00m = 22 Oct 21:50 UTC+9 = 22 Oct 14:50 UTC+2 arr FRA\nC1 FRA: L2 dep 23 Oct 07:25, gap 16h35m (995 min) > 720: long_layover."}
]}

3. Describe the format for the teacher. Write how the reasoning should look, and how long it may be, in synthetic_data_generation_instructions in the job description. That text goes into every prompt the teacher uses to write training examples and their reasoning, and never into the student’s prompt, so it costs nothing per request. The instruction we used for the flight task is quoted in the section on reasoning length below.

The teacher then writes every synthetic example together with its reasoning in that format, and fills in reasoning for any seed example that has none. The student learns to write the working first and the answer after it, and the evaluation scores only the answer. With the flag off, the same dataset trains a student that answers directly, which is how we trained the one-pass models above, so you can compare both versions of your own task on identical data.

Reasoning training works with Qwen3 (0.6B, 1.7B, 8B), Qwen3.5 (0.8B, 2B, 4B, 9B), Qwen3.6-35B-A3B and Nemotron-3.5-Lightning-30B-A3B students. The full reference is on the reasoning models docs page.

What happens with reasoning

Qwen3.5-4B	Invoice	Flight
Untuned, thinking off	53 ± 5.0	13 ± 3.4
Untuned, thinking on	37 ± 4.8	0
Fine-tuned, thinking off	84 ± 3.7	45 ± 5.0
Fine-tuned, thinking on	98 ± 1.4	95 ± 2.2
Teacher	97 ± 1.7 (GLM 5.3)	100 (Kimi K3)

Exact match of the JSON answer on 100 test cases per task, ± binomial standard deviation.

With reasoning on, the fine-tuned 4B scores around its teachers: 98 against 97 for GLM 5.3 on invoices, and 95 against 100 for Kimi K3 on flights. GLM 5.3 has 744B parameters and Kimi K3 2.8T, 186x and 700x the size of the student. A student this much smaller can land level with its teacher because the training data is validated before training and the student spends all of its capacity on one task.

This is the reasoning the student writes for an invoice whose three lines add up to 8,990.00 while the invoice asks for 9,017.00:

Check 1: invoice says PO-54311, PO is PO-54311. Same.
Check 2: first aid kits invoiced 50, received 50; barcode scanners invoiced 10, received 10; mop buckets invoiced 50, received 50.
Check 3: first aid kits PO 79.90, ceiling 81.4980, invoiced 79.90; barcode scanners PO 189.00, ceiling 192.7800, invoiced 189.00; mop buckets PO 62.10, ceiling 63.3420, invoiced 62.10.
Check 4: 3,995.00; +1,890.00 = 5,885.00; +3,105.00 = 8,990.00; no freight. Stated total 9,017.00. Mismatch. Decision: hold_total

and for the Paris connection:

L1 JFK dep 19 Oct 11:05 UTC-4 + 7h 40m = 19 Oct 18:45 UTC-4 = 20 Oct 00:45 UTC+2 arr CDG
C1 CDG: L2 dep 20 Oct 01:45, gap 1h00m (60 min) < MCT 90: short_connection.

Both stop as soon as the answer is known. The invoice model’s two errors are slips in the running sum on totals in the thousands. The flight model’s five errors are mostly valid itineraries it called long_layover, which is where the gap to Kimi K3 comes from.

How long should a small model think?

Reasoning is paid for in output tokens, and off-the-shelf reasoning models do not keep it short. Chen et al. measured this across eight frontier reasoning models in The Price Reversal Phenomenon (2026): thinking tokens are the dominant cost component for nearly all of them and are billed at the output rate, and their number varies enormously, from 562 thinking tokens for GPT-5.2 to more than 11,000 for Gemini 3 Flash on the same AIME problem.

The untuned 4B shows the same behaviour at small scale. With thinking on and the platform’s default budget of 2,048 completion tokens, it runs out of budget before answering on 63 of the 100 invoices and on all 100 itineraries. In most of these traces it already has the answer and keeps checking: one invoice trace reaches “Label: hold_quantity” and then asks itself whether “PO number referenced” includes the prefix “PO-” until the budget runs out. The 37 invoice answers it did finish were all correct, at a median of 1,253 reasoning tokens.

The trained model reasons for a median of 272 tokens on invoices (507 at most) and 208 on flights (396 at most). That length comes from the format of the seed examples and one paragraph in synthetic_data_generation_instructions. For the flight task it reads:

Reasoning: one line per leg, adding the duration to the local departure and converting the result to the arrival airport’s UTC offset; then one line per connection with the next departure, the connection time in minutes, and the comparison with the MCT and with 720. Stop at the first failing connection. No prose, no restating the policy, no revisiting a finished connection.

Since thinking tokens are billed like output tokens, the cost follows directly from the length. We measured the invoice model on one H100 at USD 0.108 per million output tokens, which comes to USD 0.028 per 1,000 decisions at about 263 output tokens per request. On the same hardware, the untuned model’s 2,048-token budget costs at least USD 0.22 per 1,000 requests, and 63% of those requests end without an answer. On a hosted model at Gemini 3 Flash’s listed output price from the paper (USD 3 per million), the 785 reasoning tokens our Kimi K3 teacher spends per itinerary come to USD 2.36 per 1,000 requests, about 80x the trained 4B. The last two figures are arithmetic on listed prices and our token counts, not measured bills.

Try it

Everything behind these numbers is in the benchmark repo: seeds, configs, training data and predictions, with the distil commands to rerun every model. The four models are on Hugging Face: invoice with reasoning, invoice without, flight with reasoning and flight without.

If you have a task where a small model gets the easy cases right and the multi-step ones wrong, sign up, set enable_thinking: true, and describe the reasoning you want in synthetic_data_generation_instructions. If you are not sure your task needs reasoning, train it both ways from the same dataset and let the evaluation decide.

---

distil labs trains task-specific small language models that are cheaper and faster per request than general-purpose LLMs, with equal-or-better accuracy on bounded tasks. Docs · GitHub · Slack