What Is Context Engineering?
Context engineering is the discipline of designing the model's entire field of view at the moment it acts: instructions, retrieved documents, tool definitions, state files, and memory, not just the prompt. A model knows nothing about your business beyond its context window, so the system that fills that window decides the quality of the output.
Prompt engineering optimizes a single message. Context engineering designs the model's entire field of view at the moment it acts, what Anthropic's engineering team describes as curating the optimal set of tokens the model sees at inference time. A language model has no knowledge of your business except what is in its context window right now. It does not remember last week. It does not know your file structure, your tone rules, or the decision your team made on Tuesday, unless something puts that information in front of it.
Once you see it that way, the question changes. It stops being "how do I phrase this request?" and becomes "what does the model need to see to do this job, and what is the system that reliably puts it there?"
What Counts as Context?
Context covers six input types: standing instruction files, retrieved documents, tool definitions, state files, persisted memory, and the conversation history itself. Each one shapes the model's behavior, and each one can be designed deliberately. Most teams have only ever edited the first few words of the first one.
When we audit an underperforming AI system, we map every input the model receives. The list is always longer than the team expects:
- Instruction files: the standing rules, constraints, and verification commands the model loads before any task.
- Retrieved documents: whatever search or RAG pipeline decides is relevant, including its mistakes.
- Tool definitions: the names, descriptions, and parameters of every tool, which quietly teach the model what it can and cannot do.
- State files: progress logs, decision records, and task lists that carry work across sessions.
- Memory: summaries and notes persisted from earlier conversations, correct or not.
- Conversation history: everything said so far, including dead ends the model may treat as instructions.
Every one of those is a design surface. The same map applies whether the system is a coding agent or the workflow automation behind a back office: the inputs nobody designed are where reliability leaks out.
How Does the File-State Pattern Work?
If a fact matters, it lives in a file in the repository, not in chat history and not in anyone's head. An instructions file carries durable rules, a progress file carries position, and a decisions file carries reasoning. A fresh session must be able to rebuild its bearings from those files alone.
The most useful pattern we run is simple: if a fact matters, it lives in a file in the repository, not in anyone's head and not in the chat history. The repo is the spec. A standing instructions file at the root carries the project overview, the hard constraints, and the exact commands that verify the work. Progress and decisions get written to dedicated files at the end of every working session.
The test we apply is the fresh-session test. Open a brand new session with no history and ask the agent to continue the work. If it cannot rebuild its bearings from the files alone in a few minutes, the context design has a hole. Chat history is not infrastructure. Files are.
The division of labor between files matters too. The instructions file carries durable rules: stack, constraints, how to verify work. The progress file carries position: what is finished, what is in flight, what comes next. The decisions file carries reasoning: what was chosen, what was rejected, and why, so the same debate does not get relitigated by a model that was not in the room. Mix those three up and every file becomes a junk drawer nobody trusts. Anthropic's own Claude Code guidance applies the same discipline to its instructions file: keep it short and prune it ruthlessly, because a bloated file causes the model to miss the rules that matter.
This is the same discipline we install when we build custom AI agents for clients: the agent's world is defined in files the client can read, audit, and change. Context is one of five subsystems in the larger reliability doctrine we cover in our harness engineering field note.
Why Is Context the Highest-Leverage Layer?
Because the same model that fails a task with poor context often completes it cleanly once the context changes, and nothing else did. A context fix is a one-time edit to a versioned file you own. A model upgrade raises your bill on every call, forever. Prove the context right before blaming the model.
We have watched the same model fail a task repeatedly, then complete it cleanly after the context changed and nothing else did. No bigger model, no clever phrasing. The difference was a constraints file it could actually find, a verification command it could actually run, and state it could actually read.
That experience repeats often enough that we treat it as a rule: before you blame the model or pay for a bigger one, prove the context was right. Capability is not the bottleneck nearly as often as people assume. Visibility is.
There is also a cost asymmetry worth naming. A model upgrade raises your bill on every single call, forever. A context fix is a one-time edit to a file you own, it is versioned in git, anyone on the team can read it, and it keeps paying off when the next model arrives. Cheap, auditable, durable. That is why context is the first place we look and the last thing we give up on.
What Are the Most Common Context Failure Modes?
Three failures dominate: context rot, where instruction files bloat until key rules sit in the middle where models attend worst; stale memory, where outdated notes drive confident wrong answers; and silent retrieval misses, where the pipeline returns the wrong document and nobody notices. All three are predictable, and all three are fixable.
Context fails in predictable ways, and the research backs the patterns we see in client work. The Lost in the Middle study found that a model's accuracy on a document question task can drop by more than 20 percent just by moving the relevant document to the middle of the context, in the worst case scoring below the 56.1 percent the same model managed with no documents at all. Position is not a detail. Three failure modes show up constantly:
- Context rot. Instruction files grow by accretion until the important rules are buried in the middle, where models attend worst. The fix is structural: keep the entry file short, a router of 50 to 200 lines, and split detail into topic files loaded on demand.
- Stale memory. A persisted note says the API uses version 2, the API moved to version 3, and the model confidently builds against the wrong one. Memory needs an owner and a review cadence, like any other dependency.
- Silent retrieval misses. The RAG pipeline returns the wrong document and the model never says so, because it cannot know what it did not see. You only catch this by logging what was retrieved alongside what was produced.
| Finding | Evidence | Source |
|---|---|---|
| Models attend worst to the middle of long contexts | GPT-3.5-Turbo's multi-document QA accuracy drops by more than 20 percent as the relevant document moves position; in 20 and 30 document settings, worst-case performance falls below its 56.1 percent closed-book score | Liu et al., Lost in the Middle (TACL 2024) |
| Recall degrades as the context window fills | As token count grows, a model's ability to accurately recall information decreases, a gradient Anthropic calls context rot | Anthropic, Effective Context Engineering for AI Agents (2025) |
| Bloated instruction files get ignored | Anthropic's guidance for its own coding agent: keep the instructions file concise, because overlong files cause the model to lose the rules that matter | Claude Code documentation, Best Practices |
Catching these failures is an evaluation problem as much as a context problem. Our note on AI agent evaluation covers the discipline that surfaces them: a checker separate from the worker, and tests that refute "it works" instead of confirming it.
How Do We Apply Context Engineering in Client Work?
On every engagement, context work comes before model work. We write the instructions file first, define the verification commands second, and wire state persistence third. Only then do we tune prompts, and by that point there is usually little tuning left to do. Most reliability complaints trace back to a context layer nobody designed.
On every engagement, context work comes before model work. We write the instructions file first, define the verification commands second, and wire state persistence third. Only then do we tune prompts, and by that point there is usually little tuning left to do. The same sequence drives our AI consulting diagnoses: most "the AI is unreliable" complaints trace back to a context layer nobody designed on purpose. It is also how a system like our AI receptionist stays on script across thousands of calls: the rules live in files it loads every time, not in anyone's memory of a prompt that once worked.
If your AI system is inconsistent, do the cheap experiment before the expensive one. Write down everything the model sees. Fix the worst gap. Run the task again and compare. Only after the context is provably right do we let anyone blame the model, and by then there is usually nothing left to blame.
