What Is an Instruction File Like CLAUDE.md or AGENTS.md?
CLAUDE.md and AGENTS.md are markdown files checked into a repository that an AI coding agent reads before starting any task. AGENTS.md is an open format adopted by over 60,000 GitHub repositories and 28-plus coding tools; Claude Code reads its own CLAUDE.md instead.
An instruction file is a plain markdown file, committed to git, that an AI coding agent reads automatically at the start of a session. It carries the project overview, the commands that build and test the code, and the hard rules the agent must not break. AGENTS.md is the open, tool-agnostic version of this idea, now used in more than 60,000 open-source repositories and natively supported by over two dozen coding tools including Cursor, Codex, and Gemini CLI. Claude Code reads its own CLAUDE.md file instead, which does the same job. The two formats differ only in filename, so everything in this report applies to both.
Why Do Agents Ignore Rules That Are Written Down?
Agents can only follow rules they can read, and they have no access to Slack, tribal knowledge, or a decision made verbally last week. A rule that is not written in the repository does not exist for the agent, which is why adding rules to one file feels productive even as it stops working.
An agent has exactly three inputs when it starts a task: the system prompt, whatever files it reads, and whatever a tool returns. It cannot see your Slack thread, your Jira ticket, or the conversation you had with a teammate on Friday, so information that only lives in someone's head does not exist for the agent. That is why the usual reaction to a mistake, adding a rule to the instruction file, feels correct. Writing the rule down is the right instinct, and the mistake is in where it gets written.
Why Does a Bigger Instruction File Make an Agent Worse?
Past a certain size, an instruction file stops helping because models do not read a long context evenly. Long-context research shows a U-shaped attention curve: models handle information at the start and end far better than the middle, so a hard rule buried at line 300 is frequently ignored even though it is written down.
Adding more rules to one growing file does not make an agent more reliable. Past roughly 300 to 600 lines, three things happen at once. The first is instruction bloat: a large file eats 10 to 20 percent of the context window before the agent has read a single line of actual code, crowding out the reasoning space a complex task needs. The second is a documented model behavior called lost in the middle. Liu et al. (2023) tested how well language models retrieve information depending on where it sits in a long context, and found a consistent U-shaped curve: models handle information placed at the very start or very end of the context far better than information buried in the middle, and the effect held even for models built specifically for long-context tasks. A hard security rule sitting at line 300 of a 600-line file is the kind of instruction that finding proves gets ignored.
| Position of the answer in a 20-document context | GPT-3.5 accuracy |
|---|---|
| Document 1 (start of context) | ~75% |
| Document 10 (middle of context) | ~55% |
| Document 20 (end of context) | ~72% |
The third is priority conflict. A hard constraint like "never run destructive commands" and a soft preference like "prefer short functions" end up formatted identically, so the agent has no signal for which one is a red line. All three problems get worse over time for the same reason documentation bloat does anywhere: nobody deletes old rules, because deletion always feels riskier than addition, so the signal-to-noise ratio drops every week.
What Is the Router Pattern for Instruction Files?
The router pattern keeps the entry file to 50 to 200 lines: a project overview, first-run commands, a short list of hard constraints, and one-line pointers to topic docs. Detailed rules live in separate files that the agent opens only when a task needs them.
This approach is called progressive disclosure, and it treats the entry file as a router rather than an encyclopedia. AGENTS.md or CLAUDE.md stays short: what the project is, how to run it, how to verify it, and no more than about fifteen hard, non-negotiable constraints. Everything else moves into topic docs of 50 to 150 lines each, kept in the module or directory they apply to, such as a database rules file next to the database code and an API patterns file next to the API layer. The router links to each one with a single line saying when it applies, so the agent pulls a doc in only when the task touches that area. We install this structure as part of every harness engineering build, because an agent's context window is a scarce resource and should not be used to store a whole company's tribal knowledge.
Matt Pocock, who has published some of the most widely used practical guidance on this pattern, describes the entry file the same way: a small, focused file that points elsewhere, giving the agent just enough to start and letting deeper context load on demand through nested files or skills. He also gives a budget, noting that frontier thinking models follow roughly 150 to 200 discrete instructions with real consistency and that smaller or non-reasoning models handle fewer. Treat that number as a hard ceiling on your entry file rather than a target to fill.
A 2026 study out of King's College London and the University of Passau measured the effect directly, running the same coding agents with and without an AGENTS.md file across 10 repositories and 124 real pull requests. With the file present, median agent runtime fell by 28.64 percent and output token consumption by 16.58 percent, while task completion stayed comparable. A well-scoped instruction file speeds the agent up in addition to preventing mistakes.
| Metric (with AGENTS.md vs. without, same tasks) | Measured effect |
|---|---|
| Median agent runtime | 28.64% lower |
| Output token consumption | 16.58% lower |
| Task completion behavior | Comparable, no regression |
Why Should Instructions Follow One Concept Per File?
One concept per file means a task pulls in only the context it needs. A topic doc that mixes deploy steps, style preferences, and one specific bug fix from last quarter forces the agent to load all three every time it needs any one of them.
Without the one-concept-per-file rule, progressive disclosure just relocates the same bloat into more files. If your database rules file also contains deploy instructions and a paragraph about a WebSocket bug from March, an agent working on a schema migration loads all of it, including the two-thirds that has nothing to do with the task in front of it. Split by concept, with one file for database constraints, one for API patterns and one for deploy steps. A note about a past bug isn't an instruction, so turn it into a test case that fails if the bug comes back instead of leaving a paragraph every future session has to read.
How Do You Know If Your Instruction File Is Actually Working?
You find out with a fresh-session test: open a brand-new agent session with nothing but the repository and check whether it can answer five basic questions about the project without guessing. Each blank points to a gap in the repository's documentation rather than a weakness in the agent.
Open a completely fresh agent session, point it at the repository and nothing else, then check whether it can answer five questions on its own: what is this system, how is it organized, how do I run it, how do I verify it, and where are we right now. Every blank answer marks a spot where the agent has to guess, and a wrong guess becomes a bug that a human then has to trace back to a documentation gap. The test costs about as much as writing one prompt, and it is the single fastest way to find out whether your instruction file works as a router or has become a long list of rules.
Why Should Instructions Live in the Repo Instead of a Wiki?
Instructions belong in the repository because it is the one place an agent reads by default at the start of every session. A wiki page, Notion doc or Slack thread is invisible to the agent unless something puts it in front of the model, so the repository is the only reliable source of truth for a working system.
Confluence pages go stale because updating them is optional and disconnected from the code. A rule in the repository, next to the code it governs, gets updated in the same commit as the code change that made it necessary, or it visibly goes stale where anyone can see it happened. For an agent, an instruction in the repo is the only form of documentation it can act on, since a wiki link the agent never opens carries the same practical weight as a rule that was never written down. We structure state and progress files the same way in every custom AI agent we build: if something isn't in a file the agent will read, we don't count on it.
The benefit compounds in a long-running autonomous coding loop, where an agent working overnight without a human in the room has nothing but the repository to orient itself session after session. A router-style instruction file is what keeps that unattended run from drifting.
