What a context window is, and why bigger isn't always better for accuracy
A model's context window is the maximum amount of text (measured in tokens, not words or characters) it can consider in a single request — the prompt, any documents attached, the conversation history, and the response it generates, all counted together against one ceiling, commonly expressed as a number like 200,000 or 1,000,000 tokens depending on the model. A bigger number sounds like a strictly better one. In practice, dumping more into a large context window doesn't reliably produce a more accurate answer, and can make things worse.
What a large context window is genuinely good for
Feeding in an entire document, a large codebase, or a long conversation history without needing to manually chunk or summarize it first — tasks that are fundamentally about having access to a lot of information at once. A 1,000,000-token context window (common on current frontier models, roughly 700,000+ words depending on the content) can hold a very large document or many files in one request, which a much smaller window — say 32,000 tokens, common on older or smaller models — simply couldn't fit no matter how relevant the content is.
Why more context isn't the same as better accuracy
Two distinct problems show up as context grows, independent of whether the model window technically has room for it all:
Signal dilution. If the actually-relevant information is a small fraction of a huge prompt, the model has to find it among a lot of irrelevant surrounding text. Retrieval across a long context is not perfectly uniform in practice — content in the middle of a very long input can get less effective attention than content near the beginning or end, a pattern often called "lost in the middle." Padding a request with tangentially related material doesn't just cost tokens, it can measurably reduce accuracy on the part that actually mattered.
Stale or conflicting information. A long conversation or a large dumped document can contain outdated statements, corrections that supersede earlier text, or genuinely contradictory information from different sources. A model has to reconcile all of that within the same context, and more raw volume means more opportunity for something contradictory to be sitting in there unnoticed, quietly increasing the chance of an answer that's technically grounded in the input but wrong because the input itself was inconsistent.
The practical implication
The better question isn't "how much can I fit," it's "how much of what I'm including is actually relevant to this specific request." Retrieving and including only the relevant sections of a large document usually outperforms dumping the entire document in, even when the entire document would technically fit — not because of a hard technical limit, but because a tighter, more relevant context tends to produce more accurate output than a larger, noisier one. Context window size determines what's possible to include; it doesn't determine what's wise to include.
Sources: general behavior of long-context language models regarding position-dependent retrieval effectiveness ("lost in the middle" pattern) and the distinction between context capacity and retrieval precision, as discussed in current LLM API documentation and research on long-context evaluation.