What Is a Context Window?

The context window is the maximum number of tokens a model can hold in a single forward pass. It’s a hard architectural ceiling, not a guideline, and everything competes for it: system prompt, retrieved passages, tool schemas, conversation history and the generated answer all draw on one budget.

What counts against the window?

More than people usually budget for. Output is the item most often forgotten.

Occupant	Counts?	Behaviour
System prompt	Yes	Paid on every single call
Tool schemas	Yes	Fixed, and often large
Retrieved context	Yes	Usually the biggest share
Conversation history	Yes	Grows every turn
Generated output	Yes	Reserved from the same total

distil labs reflects that last row directly: teacher_max_tokens defaults to 32000 in the configuration file, documented as deliberately below typical model limits so the reserved output budget doesn’t crowd out large prompts.

What happens when you exceed it?

You get an error or silent truncation, depending on the serving runtime, but never a graceful summary. Truncation is the worse outcome, because the model answers confidently from whatever survived. This is also why the window interacts with memory: everything inside it has entries in the KV cache, so a longer window raises your peak VRAM footprint even when the weights are unchanged.

Does a longer window mean better answers?

Not reliably. Liu et al. found in Lost in the Middle that accuracy is highest when the relevant information sits at the beginning or end of the input and degrades significantly when models must retrieve it from the middle, including on models built for long contexts.

Place what matters at the edges, and prefer retrieving less. See the generation guide for how limits surface in code, and what is a small language model for how window size factors into picking a student.