TL;DR: A model uses learned parameters together with the context supplied on this call. Current task evidence must be supplied or fetched; it is not automatically present in the weights. Context engineering is deciding what goes in: instructions, examples, retrieved documents, tool results, history and remembered facts, in what order and at what size. Every token costs money and time and competes for the model's attention, so the goal is the smallest set of information that gets the task right, assembled by a rule you can test.
From prompt to context
Prompt engineering asks how to word an instruction. Context engineering asks a wider question: of everything the system could show the model on this call, what should it see? For a single question to a chatbot the two are nearly the same. For an agent working through a long task, the instruction is a few hundred tokens and the rest, tool outputs, files, history, can run to tens of thousands. That rest decides most of the behaviour.
Anthropic's engineering guidance describes context as a finite resource with diminishing returns, an "attention budget", and frames the job as finding "the smallest possible set of high-signal tokens" for the outcome you want (Anthropic, as of October 2026).
What competes for the window
| Source | Typical risk if unmanaged | Common control |
|---|---|---|
| Instructions | Grow with every incident until they contradict each other | A short brief plus documents loaded only when needed |
| Retrieved documents | Too many, partly relevant passages | Retrieve fewer, rerank, cite |
| Tool results | Huge raw outputs (a full web page, a 500-row table) | Trim or summarise before returning to the model |
| History | Long conversations crowd out the task | Summarise older turns, keep decisions as records |
| Memory | Stale or wrong facts persist | Provenance, expiry, user correction |
Why more context is not safer
Three costs grow with context. Money: input tokens are billed on every call, and agents make many calls. Latency: longer inputs take longer before the first token. Quality: research on long contexts has found that models can use information in the middle of a long input less reliably than information near the start or end, and irrelevant passages can mislead. A team that "just adds everything" often gets slower, costlier and worse answers.
A worked budget
A support assistant answers billing questions from a 200-page policy. Putting the whole policy in every call is possible on a large window but wasteful. Teaching fixture:
| Part | Tokens |
|---|---|
| Instructions and output format | 800 |
| Customer's plan and last three invoices | 600 |
| Five retrieved policy passages | 2,500 |
| Last six conversation turns, older ones summarised | 1,200 |
| Total per call | 5,100 |
Compared with about 110,000 tokens for the full policy, that is roughly 95% fewer input tokens per call. At an illustrative 2 USD per million input tokens and 100,000 conversations a month with four calls each, the difference is about 88,000 USD against 4,100 USD a month in input cost, before the quality and latency effects. Caching a stable prefix narrows the gap, which is why the stable parts belong first (see prompt and prefix caching).
What gets priority
Reserve the applicable output and internal-reasoning budget first, then fit the input within the endpoint's rules. Keep mandatory instructions and the current user request; include authorized, current records needed to answer it. Trim redundant tool output and old conversational detail before removing facts that control the decision. A memory or summary must not override an explicit current instruction.
This is a product-specific allocation, not a universal positional ranking. Instruction authority and source trust matter as well as token order. When the input cannot fit safely, retrieve a narrower set, ask for scope or report the limit. Test that compaction preserves constraints such as “do not send.”
Patterns for long tasks
- Just-in-time retrieval. Keep lightweight references (file paths, record IDs, links) and let the agent fetch details when it needs them, instead of loading everything up front.
- Compaction. When a conversation nears the limit, summarise it and continue with the summary, keeping decisions and open items.
- Notes outside the window. The agent writes progress and decisions to a file or record it can reread, so they survive compaction.
- Sub-tasks with their own context. A sub-agent explores a large codebase or document set and returns a short result, keeping the main context clean.
Each pattern trades something. Summaries lose detail; just-in-time retrieval adds calls and latency; sub-agents add cost. The decision is which loss the product can afford.
The product decisions
A PM owns three questions here. What must the model always see (the rules that keep users safe)? What may it see only when relevant (most documents)? What must it never see (other customers' data, secrets)? The third is enforced outside the model, in retrieval and permissions (see permission-aware RAG). The first two become a written context recipe per task, with a token budget, that the evaluation set can test.
In the interview
Context engineering shows up when a loop asks why an assistant gets worse in long conversations, how to cut inference cost, or how to design an agent that runs for hours. The answer that scores names what goes into the window and why, gives a token budget with arithmetic, and picks a pattern for long tasks with its trade-off. "Use a model with a bigger context window" without a budget scores poorly.