A context window is just a very long shelf.

“We have a huge context window now — can’t we just dump the whole client wiki in?” The answer is yes, and also, not quite. A context window is real room to work with, and it keeps growing. Whether the model reads everything you put there with the same care is a separate question, and the answer there is no.

What exactly is a context window?

Anthropic's own documentation puts it plainly: the context window is all the text a language model can reference when generating a response, including the response itself¹. That covers the system prompt, every message in the conversation, any document you hand it, and the reply it writes back. Training is different: the huge pile of text the model learned from, long before your call, is fixed knowledge it already has. The context window is what it holds open in front of it for this one exchange.

How big is “huge” right now?

The number keeps climbing. Anthropic's Claude Opus and Sonnet models now carry a 1M-token context window², with its smaller Haiku model still at 200K tokens². Google says Gemini was the first model capable of accepting 1 million tokens⁴, and OpenAI's newest GPT-6 line lists 1.05 million tokens⁵ as standard. A window that size can hold a long client brief, a full style guide, and a stack of past correspondence, in the same call, with room to spare.

Does a bigger window mean it reads every part the same?

Not on its own. Anthropic's own engineering team names the failure mode directly: as the number of tokens in the context window increases, the model's ability to accurately recall information from that context decreases³, a pattern they call context rot¹. Part of the reason is architectural. A transformer lets every token weigh every other token, so the work involved grows with the square of the length: Anthropic describes n² pairwise relationships for n tokens³. Doubling the window roughly quadruples the strain. Underneath, as Anthropic's engineers put it, LLMs have an "attention budget" that they draw on when parsing large volumes of context³.

What's Real?

• •Position beats volume. Studying real long-context models, researchers found performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts⁶. Where a fact sits in the window can matter more than whether it's in there at all.
• •One buried fact is manageable. Several are harder. Google's own long-context guidance warns that in cases where you might have multiple 'needles' or specific pieces of information you are looking for, the model does not perform with the same accuracy⁴. A single fact tucked into a huge prompt is a fair test. A client brief with a dozen buried facts is the real job, and a harder one.
• •The usable number is often smaller than the advertised one. NVIDIA's RULER benchmark tested seventeen long-context models and found that while these models all claim context sizes of 32K tokens or greater, only half of them can maintain satisfactory performance at the length of 32K⁷. A spec sheet only says what a model can hold.
• •Every token in the window still costs money. A bigger prompt costs more to send whether or not the model reads each part with care, and the token brief⁸ and the caching brief⁹ both apply here at full strength. Filling the window because you can is a cost decision as much as a design one.

The “so what” — for anyone serving tech clients

• •Ask what “context window” is being sold as. A vendor pitching “1 million tokens” is describing capacity. The useful follow-up is how it performs on documents like the client's. That's the difference between a spec and a result.
• •Put the fact that has to land where it gets read. If a brief, a system prompt, or a retrieved document has one point that matters most, place it near the start or the end of what the model sees. It costs nothing and it's worth doing on every call.
• •Don't let capacity replace retrieval. A model that can hold a million tokens can still be pointed at the wrong ones. Retrieval¹⁰ that fetches the right handful of documents usually beats a habit of pasting in everything and hoping. The window got bigger. Choosing what goes in it is still the job — it just got easier to skip.