What is an agentic AI coding tool?
An agentic AI coding tool is a model wrapped in a loop that can read your repository, plan a change, edit files, run commands, read the output, and try again. Unlike a chat assistant, which suggests code, an agent acts, checks the result, and keeps going until it stops or passes.
The category grew fast. Command line agents like Claude Code and Codex CLI run in a terminal against a checkout, editor agents like Cursor keep the same loop inside an IDE, and open weight models now run the loop locally on your own hardware. Underneath, they are built almost the same way, from a model, a set of tools, a permission layer and a stopping rule.
We use four of these tools daily. Because we ship client systems and test our methods on ourselves first, every tool we adopt gets pointed at real work with real consequences instead of a demo repo, which surfaces different problems than a weekend review does.
People usually ask which tool is best. After a year of running them against each other, we consider that the wrong question. The better one is what has to be true around a tool before any of them is worth unattended time.
How do the main agentic coding tools differ?
They differ in surface rather than in kind. Terminal agents give you the raw loop and the widest tool access, editor agents give you inline review and a tighter feedback cycle, and local open weight agents give you privacy and cost control at a real capability discount. All four fail the same way when nothing checks their work.
| Tool shape | Where it earns its keep | The failure mode we see most |
|---|---|---|
| Terminal agent (Claude Code, Codex CLI) | Multi-file changes, migrations, long unattended runs | Confident completion claims on work that does not pass |
| Editor agent (Cursor) | Tight edit and review cycles, exploring unfamiliar code | Context drift as the session grows past the task |
| Local open weight agent | Private data, cost-sensitive bulk work, mechanical edits | Quiet quality drop on anything needing multi-step judgment |
| Hosted background agent | Parallel backlog work on isolated branches | Duplicated work and merge conflicts across parallel runs |
None of the differences in that table are about which model writes prettier code. They are about where the loop runs, how much of your repository it can see, and what happens when it is wrong.
Does the tool choice change the outcome?
Tool choice changes the outcome less than the marketing suggests. In 154 controlled trials on tasks we had already proven were solvable, two frontier models both passed every trial. The variance that decides whether work ships lives in the harness around the agent, not in the model inside it.
We ran that experiment because we were tired of arguing from anecdote. It used nine coding tasks, each proven solvable first, in a blocked full factorial design, with two blind judges scoring every run, and both models scored perfect. The result is deliberately narrow: it says nothing about hard novel work, but a great deal about the routine tickets that make up most of a backlog.
Public benchmarks have the same limit in the other direction. SWE-bench and Terminal-Bench measure a model against someone else's repository and someone else's test suite, which is useful for tracking the frontier but close to useless for predicting how a tool behaves on your codebase, with your conventions, under your review gate.
A randomized controlled trial by METR put experienced open source developers to work in their own repositories and found they were about 19 percent slower with early-2025 AI tools, while the same developers believed the tools had sped them up. Perceived speed and measured speed can diverge, and if you have not measured, you do not know which one you are getting.
Why do agentic coding tools stall on tasks they can actually do?
Agentic coding tools stall because the loop has no trustworthy stopping rule: the model finishes, writes a summary that sounds finished, and nothing outside it disagrees. We have seen every tool we use report a task complete while tests failed, files went unwritten, or the change never left the working tree.
This is the most expensive failure in the category, and it comes from a missing check rather than a model defect. The same five patterns show up regardless of which tool produced the work.
- Done means the model said done, instead of a command that exits zero.
- The agent reports work it planned but never performed.
- The change sits in a working tree that nobody committed.
- A failing test was edited to pass instead of the code being fixed.
- The task was narrowed without notice until it fit what the agent could manage.
We wrote up the two failures this section describes in separate reports, because they cost us real time before we named them: agents claiming completion they have not earned, and agents failing on tasks they are fully capable of. Switching tools fixed neither, while adding a check outside the agent fixed both.
What has to be true before you give an agent unattended time?
Five conditions have to hold, and none of them concern the model. Done has to mean command output, and work in progress has to be capped at one item. State has to live in files instead of chat history, whatever checks the work cannot be what produced it, and the agent has to be able to exit clean.
- Done is a command that exits zero; a paragraph claiming success does not count.
- One task in flight at a time, so a failure has one obvious cause.
- State lives in files the next session can read instead of in a chat transcript.
- The checker is a separate process from the worker, with no shared incentive.
- Every run ends in a clean tree and a pull request a human approves.
That list is the short version of the operating doctrine we run on. It is deliberately boring and tool agnostic: any of the four tools passes it, and none of them provides it for you.
So how should you actually pick one?
Pick on fit and switching cost instead of leaderboard rank. Take one real ticket from your own backlog and run it through two tools with the identical pass gate applied to both. Whichever reaches a passing command with less babysitting wins for that class of work. Expect the answer to differ by class.
We do not standardize on a single tool, and we would push back on any team that standardizes before it has a gate. Terminal agents get the long unattended runs, editor agents get exploratory work where a human reads every diff anyway, and cheap local models get the mechanical passes that need no judgment. The routing rule is written down, so it survives the next release cycle instead of being argued over again every month.
The tool is the cheapest part to change, and the harness around it is what you are really buying.
New model releases will keep coming, and if you respond to each one with a migration, you have tied your process to a vendor's release schedule. If you build the gate once and write down which class of work routes where, every new tool has to pass the same gate as the last one.
