What is an agentic AI coding tool?
An agentic AI coding tool is a model wrapped in a loop that can read your repository, plan a change, edit files, run commands, read the output, and try again. The word that matters is loop. A chat assistant suggests code. An agent acts, checks the result, and keeps going until it stops or passes.
The category grew fast. Command line agents like Claude Code and Codex CLI run in a terminal against a checkout. Editor agents like Cursor keep the same loop inside an IDE. Open weight models now run the loop locally on your own hardware. The shape is nearly identical across all of them: a model, a set of tools, a permission layer, and a stopping rule.
We use four daily. Our studio ships client systems and our lab tests the methods first, so every tool we adopt gets pointed at real work with real consequences instead of a demo repo. That is a different sample than a weekend review, and it surfaces different things.
The buying question people ask is which tool is best. After a year of running them against each other, that is the wrong question. The better one is what has to be true around the tool before any of them are worth unattended time.
How do the main agentic coding tools differ?
They differ in surface, not in kind. Terminal agents give you the raw loop and the widest tool access. Editor agents give you inline review and a tighter feedback cycle. Local open weight agents give you privacy and cost control at a real capability discount. All four fail the same way when nothing checks their work.
| Tool shape | Where it earns its keep | The failure mode we see most |
|---|---|---|
| Terminal agent (Claude Code, Codex CLI) | Multi-file changes, migrations, long unattended runs | Confident completion claims on work that does not pass |
| Editor agent (Cursor) | Tight edit and review cycles, exploring unfamiliar code | Context drift as the session grows past the task |
| Local open weight agent | Private data, cost-sensitive bulk work, mechanical edits | Quiet quality drop on anything needing multi-step judgment |
| Hosted background agent | Parallel backlog work on isolated branches | Duplicated work and merge conflicts across parallel runs |
Notice what is not in that table. None of the differences are about which model writes prettier code. They are about where the loop runs, how much of your repository it can see, and what happens when it is wrong.
Does the tool choice change the outcome?
Less than the marketing suggests. On tasks we had already proven were solvable, we ran 154 controlled trials pitting two frontier models against each other. Both passed every trial. The variance that decides whether work ships lives in the harness around the agent, not in the model inside it.
We ran that experiment because we were tired of arguing from anecdote. Nine coding tasks, each proven solvable first, blocked full factorial design, two blind judges scoring every run. Both models scored perfect. The result is narrow on purpose. It says nothing about hard novel work. It says a great deal about the routine tickets that make up most of a backlog.
Public benchmarks carry the same limit in the other direction. SWE-bench and Terminal-Bench measure a model against someone else's repository and someone else's test suite. That is useful for tracking the frontier. It is close to useless for predicting how a tool behaves on your codebase, with your conventions, under your review gate.
There is a harder finding worth sitting with. A randomized controlled trial by METR put experienced open source developers to work in their own repositories and found they were about 19 percent slower with early-2025 AI tools, while the same developers believed the tools had sped them up. Perceived speed and measured speed are not the same variable. If you have not measured, you do not know which one you are buying.
Why do agentic coding tools stall on tasks they can actually do?
Because the loop has no honest stopping rule. The model finishes, writes a summary that sounds finished, and nothing outside it disagrees. We have watched every tool we use declare a task complete while tests failed, files went unwritten, or the change never left the working tree.
This is the most expensive failure in the category, and it is not a model defect. It is a missing check. The same five patterns show up regardless of which tool produced the work.
- Done means the model said done, instead of a command that exits zero.
- The agent reports work it planned but never performed.
- The change sits in a working tree that nobody committed.
- A failing test was edited to pass instead of the code being fixed.
- The task was quietly narrowed until it fit what the agent could manage.
We wrote both of these up separately, because they cost us real time before we named them: agents claiming completion they have not earned, and agents failing on tasks they are fully capable of. Switching tools fixed neither. Adding a check outside the agent fixed both.
What has to be true before you give an agent unattended time?
Five things, and none of them are the model. Done has to mean command output. Work in progress has to be capped at one item. State has to live in files, not in chat history. Whatever checks the work cannot be the thing that produced it. And the agent has to be able to exit clean.
- Done is a command that exits zero, not a paragraph claiming success.
- One task in flight at a time, so a failure has one obvious cause.
- State lives in files the next session can read, not in a chat transcript.
- The checker is a separate process from the worker, with no shared incentive.
- Every run ends in a clean tree and a pull request a human approves.
That list is the short version of the operating doctrine we run on. It is deliberately boring and it is tool agnostic, which is the point. Any of the four tools passes it. None of them provide it for you.
So how should you actually pick one?
Pick on fit and switching cost, not on a leaderboard. Take one real ticket from your own backlog and run it through two tools with the identical pass gate applied to both. Whichever reaches a passing command with less babysitting wins for that class of work. Expect the answer to differ by class.
We do not standardize on a single tool, and we would push back on any team that standardizes before it has a gate. Terminal agents get the long unattended runs. Editor agents get exploratory work where a human reads every diff anyway. Cheap local models get the mechanical passes that need no judgment. The routing rule is written down, so it survives the next release cycle instead of being relitigated every month.
The tool is the cheapest thing to change. The harness is what you are actually buying.
Model releases will keep landing. If your response to each one is a migration, you have coupled your process to a vendor's schedule. Build the gate once, write down which class of work routes where, and every new tool has to earn its way through the same door as the last one.
