What Is Context Engineering?
Context engineering is the practice of designing everything a model sees at the moment it acts, which covers instructions, retrieved documents, tool definitions, state files and memory along with the prompt. A model knows nothing about your business beyond its context window, so the system that fills that window decides the quality of the output.
Prompt engineering optimizes a single message, while context engineering designs the model's entire field of view at the moment it acts, which Anthropic's engineering team describes as curating the optimal set of tokens the model sees at inference time. A language model has no knowledge of your business except what is in its context window right now. It has no memory of last week, and it does not know your file structure, your tone rules or the decision your team made on Tuesday unless something puts that information in front of it.
Seen that way, the useful question is "what does the model need to see to do this job, and what is the system that reliably puts it there?" rather than "how do I phrase this request?"
What Counts as Context?
Context covers six input types: standing instruction files, retrieved documents, tool definitions, state files, persisted memory and the conversation history itself. Each one shapes the model's behavior and each can be designed deliberately, yet most teams have only ever edited the first few words of the first one.
When we audit an underperforming AI system, we map every input the model receives. The list is always longer than the team expects:
- Instruction files: the standing rules, constraints, and verification commands the model loads before any task.
- Retrieved documents: whatever search or RAG pipeline decides is relevant, including its mistakes.
- Tool definitions: the names, descriptions and parameters of every tool, which implicitly tell the model what it can and cannot do.
- State files: progress logs, decision records, and task lists that carry work across sessions.
- Memory: summaries and notes persisted from earlier conversations, correct or not.
- Conversation history: everything said so far, including dead ends the model may treat as instructions.
Each of those is a design surface, and the same map applies whether the system is a coding agent or the workflow automation behind a back office. In either case, the inputs nobody designed are where reliability leaks out.
How Does the File-State Pattern Work?
Any fact that matters is written to a file in the repository, so nothing depends on chat history or on what someone remembers. An instructions file carries durable rules, a progress file carries the current position, and a decisions file carries reasoning. A fresh session must be able to get its bearings from those files alone.
The most useful pattern we run is simple: any fact that matters goes into a file in the repository rather than staying in someone's head or in the chat history, so the repo serves as the spec. A standing instructions file at the root carries the project overview, the hard constraints and the exact commands that verify the work, and progress and decisions are written to dedicated files at the end of every working session.
We check this with what we call the fresh-session test: open a brand new session with no history and ask the agent to continue the work. If it cannot get its bearings from the files alone within a few minutes, the context design has a gap, since the work should never depend on chat history.
Each file has a separate job. The instructions file carries durable rules, such as the stack, the constraints and how to verify work. The progress file records position, meaning what is finished, what is in flight and what comes next. The decisions file records what was chosen, what was rejected and why, so a model that was not in the room does not reopen a debate that was already settled. When those jobs get mixed together, nobody can trust any of the files. Anthropic's own Claude Code guidance applies the same discipline to its instructions file: keep it short and prune it ruthlessly, because a bloated file causes the model to miss the rules that matter.
We use the same approach when we build custom AI agents for clients, so the agent's world is defined in files the client can read, audit and change. Context is one of five subsystems in the larger reliability doctrine covered in our harness engineering field note.
Why Is Context the Highest-Leverage Layer?
Because the same model that fails a task with poor context often completes it cleanly once the context changes and nothing else does. A context fix is a one-time edit to a versioned file you own, while a model upgrade raises your bill on every call from then on. Prove the context is right before blaming the model.
We have seen a model fail a task repeatedly and then complete it cleanly after the context changed, with no switch to a bigger model and no rewording of the prompt. The difference was a constraints file it could find, a verification command it could run and state it could read.
This happens often enough that we treat it as a rule: before you blame the model or pay for a bigger one, prove the context was right. Capability is the bottleneck far less often than people assume. More often, the model could not see what it needed.
The costs differ too. A model upgrade raises your bill on every call from then on, while a context fix is a one-time edit to a file you own, versioned in git and readable by anyone on the team, and it keeps paying off when the next model arrives. That is why we check the context first and keep working on it longer than any other fix.
What Are the Most Common Context Failure Modes?
Three failures come up most: context rot, where instruction files bloat until key rules sit in the middle where models attend worst; stale memory, where outdated notes lead to wrong answers; and silent retrieval misses, where the pipeline returns the wrong document and nobody notices. Each one is predictable and can be fixed.
The failure patterns we see in client work match the published research. The Lost in the Middle study found that a model's accuracy on a document question task can drop by more than 20 percent just from moving the relevant document to the middle of the context, and in the worst case it scored below the 56.1 percent the same model managed with no documents at all. Three failure modes show up constantly:
- Context rot. Instruction files grow by accretion until the important rules are buried in the middle, where models attend worst. The fix is to keep the entry file short, a router of 50 to 200 lines, and split detail into topic files loaded on demand. The term also has a narrower sense, recall degrading as the window fills on a task whose difficulty never changes, which we measured out to 730,000 tokens.
- Stale memory. A persisted note says the API uses version 2, the API has since moved to version 3, and the model builds against the wrong one. Memory needs an owner and a review cadence, like any other dependency.
- Silent retrieval misses. The RAG pipeline returns the wrong document and the model never says so, because it cannot know what it did not see. You only catch this by logging what was retrieved alongside what was produced.
| Finding | Evidence | Source |
|---|---|---|
| Models attend worst to the middle of long contexts | GPT-3.5-Turbo's multi-document QA accuracy drops by more than 20 percent as the relevant document moves position; in 20 and 30 document settings, worst-case performance falls below its 56.1 percent closed-book score | Liu et al., Lost in the Middle (TACL 2024) |
| Recall degrades as the context window fills | As token count grows, a model's ability to accurately recall information decreases, a gradient Anthropic calls context rot | Anthropic, Effective Context Engineering for AI Agents (2025) |
| Bloated instruction files get ignored | Anthropic's guidance for its own coding agent: keep the instructions file concise, because overlong files cause the model to lose the rules that matter | Claude Code documentation, Best Practices |
Catching these failures is an evaluation problem as much as a context problem. Our note on AI agent evaluation covers the discipline that surfaces them: a checker separate from the worker, and tests that refute "it works" instead of confirming it.
How Do We Apply Context Engineering in Client Work?
On every engagement we do the context work before any model work: we write the instructions file, then define the verification commands, then wire up state persistence. We tune prompts only after that, and by then there is usually little tuning left to do. Most reliability complaints trace back to a context layer nobody designed.
We follow that order on every engagement, and the same sequence drives our AI consulting diagnoses, where most "the AI is unreliable" complaints trace back to a context layer nobody designed on purpose. It is also how a system like our AI receptionist stays on script across thousands of calls, because its rules live in files it loads every time rather than in anyone's memory of a prompt that once worked.
If your AI system is inconsistent, run the cheap experiment before the expensive one: write down everything the model sees, fix the worst gap, then run the task again and compare. We only consider blaming the model once the context is provably right, and by then the problem has usually gone away.
