Open-Book vs Closed-Book QA
Open-book QA answers from a passage you supply at query time; closed-book QA answers from knowledge baked into the weights during training. Pick open-book when answers must cite a source or the corpus changes often. Pick closed-book when the knowledge is stable and you want one artefact with no retriever.
How do open-book and closed-book QA differ?
They differ in where the knowledge lives at inference time, and everything else follows from that.
| Open-book QA (RAG) | Closed-book QA | |
|---|---|---|
base.task |
question-answering-open-book |
question-answering-closed-book |
| Knowledge at inference | In the passage you pass in | In the model weights |
train.jsonl line |
messages plus a sibling context field |
messages only |
unstructured.jsonl |
Optional — realistic candidate contexts | Required — it is the knowledge source |
| Infrastructure needed | Retriever, index, chunking pipeline | None beyond the model |
| Updating the knowledge | Reindex | Retrain |
| Can cite a source | Yes | No |
| Context window pressure | High — chunks consume it | Low |
The context field is the concrete difference. In open-book data each example names the passage the answer must be grounded in, which lets the platform train the model to read rather than recall — and lets it add distractor passages so the model learns to ignore irrelevant chunks. That technique was introduced as RAFT and is exposed as synthgen.num_distractor_context_blocks in the config file.
Closed-book flips the role of unstructured.jsonl. It stops being an optional diversity aid and becomes the channel through which facts enter the model: the teacher reads your unstructured passages and fabricates question-answer pairs from them, and those pairs are what the student memorises.
What is open-book QA better at?
Provenance. A closed-book model cannot tell you which document justified an answer, because by inference time the document is gone. If a wrong answer has to be traceable — regulated advice, contract review, clinical documentation — retrieval is not optional.
Freshness. Reindexing a changed document takes minutes. Retraining takes a training run. If your corpus moves weekly, open-book is the cheaper operating model by a wide margin.
Corpus size. A retriever scales to corpora far larger than you could reasonably distil into a small student. There is no upper bound on the index; there is very much one on a 1B model’s capacity.
Measured lift on small students. Our on-device RAG study fine-tuned Llama 3.2 1B on a Siemens SIMATIC S7-1200 manual and evaluated 144 questions with an LLM judge:
| Model | Accuracy | Correct / total |
|---|---|---|
| Llama 3B base | 60.4% | 87 / 144 |
| Llama 1B base | 45.1% | 65 / 144 |
| Llama 1B tuned (weights) | 61.1% | 88 / 144 |
A 16-point lift brought the 1B to parity with a base model three times its size. That study used deepseek.v3.1 as the teacher; new work should use a current teacher such as zai.glm-5.
What is closed-book QA better at?
Operational simplicity. One artefact, no index, no embedding model, no chunking strategy to tune, nothing to keep in sync. For an air-gapped or embedded deployment that is a real advantage, not a stylistic preference.
Latency and context budget. Retrieved chunks are the largest part of an open-book prompt. Removing them shortens the input dramatically, which matters when the student has a modest context window and you are paying for every token of it in latency.
Questions retrieval answers badly. Aggregate and cross-document questions (“which of these three policies is most restrictive?”) are exactly where a top-k retriever struggles. The docs name this directly: closed-book is worth considering when a typical RAG setup does not work for you because building an effective retrieval system is difficult.
The cost is real. Every knowledge update is a retraining job, and the model will answer confidently from stale weights rather than admitting the fact has changed.
Which should you pick?
| Your situation | Pick |
|---|---|
| Answers must be traceable to a document | Open-book |
| Corpus changes weekly or faster | Open-book |
| Corpus is large relative to the student | Open-book |
| You already run a retriever for other reasons | Open-book |
| Stable knowledge, air-gapped or embedded target | Closed-book |
| Retrieval quality is the bottleneck, not the model | Closed-book |
| No infrastructure to operate an index | Closed-book |
The honest default is open-book. Most teams overestimate how stable their knowledge is, and provenance requirements tend to appear after launch rather than before. Closed-book is the specialist choice, chosen deliberately.
Can you run both?
Yes, and it is a common shape: a closed-book model that knows your domain’s stable vocabulary, fronted by retrieval for the volatile parts. Nothing in the platform prevents training two models from the same source material, and the data preparation guides for open-book and closed-book both start from the same unstructured.jsonl.
Before you build either, run teacher evaluation on the same test questions under both task types. It is cheaper than a training run and it answers the question empirically. For the wider set of options see the six task types, and for a worked open-book recipe see document QA over contracts.