Prompt engineering was step one. Now engineer the context.

“We wrote the perfect prompt, so why does the Artificial Intelligence (AI) still get it wrong?” Every team building with a Large Language Model (LLM) runs into this. The prompt is a sliver of what the model reads before it answers. The rest is context, and choosing it well has a name now: context engineering.

So what is context engineering?

Prompt engineering¹ was the first version of this skill: write a clear request, get better work back. Context engineering is the wider job. It covers everything you put in front of the model before it answers: the instructions, the examples, the documents you retrieve, the tools you expose, the running history of the chat. Anthropic calls it the natural progression of prompt engineering², and defines it as the set of strategies for curating and maintaining the optimal set of tokens (information) during LLM inference².

All of it has to fit inside the context window: the model’s short-term memory for a single task, everything it can hold in view at once, measured in tokens³. Andrej Karpathy described the goal as the delicate art and science of filling the context window with just the right information for the next step⁴. The window is finite, so the real question is which information earns a place in it.

Is more context always better?

The tempting move is to pour everything in: the whole knowledge base, every past message, all the tools at once. Two things make that backfire. A model has a limited attention budget, so Anthropic warns that context must be treated as a finite resource with diminishing marginal returns². Past a point, more text buys worse answers. Models also read the middle less carefully than the ends. Studying long-context models, researchers found that performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts, even for explicitly long-context models⁵. Bury the one fact that matters in the middle of a huge prompt and the model can walk right past it.

What's Real?

• •Context is a budget, not a bucket. A model draws on a limited attention span, and Anthropic treats context as a finite resource with diminishing marginal returns². Filling the window to the brim can lower the quality of the answer.
• •Position changes what gets used. Models handle facts at the start and end of a long input better than the ones buried in the middle, even for explicitly long-context models⁵. Where you place a fact affects whether the model uses it.
• •The fix is curation, not capacity. Anthropic’s own rule is to find the smallest possible set of high-signal tokens that maximize likelihood of desired outcome², then pull in more only when a step calls for it.
• •The name is catching on because it fits the work. Simon Willison calls context engineering a better alternative to prompt engineering⁴, closer to what the job really involves.

The “so what” — for anyone serving tech clients

• •Debug the context before the prompt. When an AI feature misfires, look at what the model could see: stale documents, missing history, ten tools where two would do. That is usually a faster fix than re-wording the request.
• •Ask vendors how they manage context. “We use AI” says nothing. A partner who can explain how they retrieve⁶ the right facts, keep the history tight, and hand the model a clean set of tools has thought about reliability. One who only talks up the model has not.
• •Budget for the plumbing. Assembling good context is real engineering, and it is where a promising demo becomes a dependable feature. It is the same gap the demo-to-production brief⁷ named: a sharp sample is a long way from a shipped product. Building the retrieval, the memory, and the guardrails is the work that ships⁸.