Qwen3.5 2B Fine-Tuning Guide

Qwen3.5-2B is the newest small student in the distil labs catalog and the first at this size with a 262,144-token native context window. It is a reasonable pick when your inputs are long, and a speculative one otherwise: no published distil labs benchmark covers the Qwen3.5 generation yet.

What changed between Qwen3 and Qwen3.5 at this size?

Two things you can act on: context length and default reasoning behaviour.

Property	Qwen3-1.7B	Qwen3.5-2B
Total parameters	1.7B	2B
Native context	32,768 tokens	262,144 tokens
Thinking mode	On by default	Off by default
Catalog value	Qwen3-1.7B	Qwen3.5-2B

The Qwen3-1.7B card documents thinking as enabled unless you pass enable_thinking=False; the Qwen3.5-2B card documents the opposite default. That flip matters more than the 300M extra parameters, because a stray <think> block in front of a JSON tool call is a parse failure, not a quality regression.

What does a 262K context window actually buy you?

Less than the headline suggests, for the tasks distil labs trains. Classification, tool calling, and closed-book QA all have short inputs; the window is irrelevant to them.

It matters for exactly one task type in the catalog: open-book QA for RAG, where the model answers from passages you supply at inference time. A larger window means you can pass more retrieved chunks without a reranking stage. That is a genuine architectural advantage over the 32K models in the catalog — Gemma 3, LFM2.5, and Qwen3 all stop at 32,768 tokens.

It is not a substitute for retrieval quality. Filling 262K tokens with weakly-relevant chunks makes answers worse and inference slower. Treat the window as headroom, not as a retrieval strategy.

Where does Qwen3.5 2B fall short?

There is no benchmark rank for it. Our published studies cover the 12-model and 15-model fields, neither of which includes Qwen3.5 — both name it as future work. Anyone quoting a Qwen3.5-2B fine-tuned accuracy figure is not quoting us.

The Qwen3.5-2B card warns about thinking loops. It notes that the 2B variant is more prone to entering thinking loops than its larger siblings at the recommended sampling settings. If you enable thinking, cap generation length.

It costs more to serve than the 1.7B it replaces. At BF16, 2B parameters of weights is 4.0 GB before you allocate any KV cache — see which SLM fits in 4GB of VRAM for the arithmetic. On a 4 GB card this model needs quantization; Qwen3-1.7B does not.

Long context is not free at inference. KV cache grows linearly with sequence length. A model that can take 262K tokens will not take them within a small memory budget.

Is there benchmark data you can rely on instead?

Yes, but it is indirect. Our closest published evidence is the mid-size MoE benchmark, where the Qwen3.6 generation tied for first after fine-tuning. That tells you the family is on a good trajectory; it does not tell you what a 2B student will score on your task.

The reliable substitute is your own teacher evaluation plus a training run. Two runs at defaults cost less than a week of speculation, and the 12-model study’s central finding — that base rank does not predict fine-tuned rank — means desk research on model cards would not have answered the question anyway.

How do you set it as your student model?

Use the exact catalog value in the base block of your config file:

base:
  task: question-answering-open-book
  student_model_name: Qwen3.5-2B
  teacher_model_name: zai.glm-5

synthgen:
  teacher_temperature: 0.6

The teacher matters more here than on a short-input task, because an open-book student is only as good as the grounded answers the teacher writes for it. openai.gpt-oss-120b is the documented default, but recent distil labs work uses GLM-5 (744B total, 40B active), which scored highest of the five teachers we compared in the traces versus synthetic benchmark. GLM is a reasoning family, so synthgen.teacher_temperature must stay between 0.5 and 0.7.

Qwen3.5 is one of the families cleared for tool-calling tasks in the supported models catalog, so tool-calling-closed-book and multi-turn-tool-calling-closed-book are both available. Leave the tuning defaults untouched on the first run so your result is comparable to the published benchmarks, then read the metrics. If it underperforms, the cheapest next experiment is the same data against Qwen3-1.7B.