Running AI Coding Agents Unattended, Without Waking Up to a Mess

What Does It Mean to Run an AI Coding Agent Unattended?

An unattended loop is an AI agent working through a backlog of tasks over hours without a human reviewing each step in real time. It only becomes safe once the harness around the model enforces limits on scope, verification, spend and approval.

Most AI coding today still has a human watching every step and approving each file change before it lands. An unattended loop removes that step for a defined stretch of time, usually overnight or across a weekend, and lets the agent work through a backlog on its own. That changes the risk as well as the convenience, because an agent that behaves well with a human glancing over its shoulder may not behave the same way once nobody is watching. The rest of this report covers how we close that gap.

How Long Can an AI Agent Actually Work Before Reliability Drops?

METR's benchmark shows AI agents completing tasks under 4 minutes almost 100% of the time and succeeding on tasks over 4 hours less than 10% of the time. The task length they can handle is doubling roughly every 7 months, which is real progress, but most agents are still not ready for long unsupervised stretches.

This number should set expectations before anyone schedules an overnight run. METR, an independent AI evaluation nonprofit, measures agent capability with a 50 percent time horizon, which is the length of task (measured by how long a skilled human would take) that a model can complete autonomously half the time. As of their March 2025 report, that horizon had been doubling roughly every 7 months for six straight years, and Claude 3.7 Sonnet's measured horizon was around one hour at the time of testing. The same study found near-100 percent success on tasks under four minutes, falling below 10 percent once a single task ran past four hours.

In practice, a single task handed to an unattended agent should be scoped small enough to sit well inside its measured reliability zone, which rules out the multi-hour epic you would give a junior engineer for a week. That is the reason for the wave-based backlogs described in the next section.

Why Do Unattended Loops Need Wave-Based Backlogs Instead of an Open Task List?

Waves keep an agent from starting six things at once and finishing none of them cleanly, as agents tend to do with an open-ended to-do list. A wave-based backlog breaks the work into small batches of independently verifiable tasks, run one wave at a time with a checkpoint between waves.

With a long, unstructured backlog, agents act like an over-eager new hire on day one. They start several things, half-finish most of them, and report broad progress that is hard to check against any of it. A wave structure groups a small number of independently scoped and independently verifiable tasks into one batch, runs only that batch, checks every task in it against a real done gate, and only then opens the next wave. Each wave fits comfortably inside the reliability window the METR data describes, and at the checkpoint between waves a human, or an automated gate standing in for one, decides whether the loop should continue.

What Is a Machine-Checkable Done Gate, and Why Does It Matter More Overnight?

A machine-checkable done gate defines done as a passing command and never accepts an agent's own claim that it finished. Agents are measurably overconfident about their own work and nobody is watching overnight, so a deterministic check or separate reviewer agent must be the only thing allowed to mark a task complete.

In an unattended run, "looks done" is the single most dangerous phrase, because no human is in the room to notice when it is wrong. Guo et al.'s widely cited calibration study found that modern neural networks are systematically overconfident, reporting higher confidence than their actual accuracy warrants, and agent behavior follows the same pattern. Ask an agent whether it finished a task and it says yes with the same confidence whether the work is excellent or broken in a way nobody has noticed. A done gate replaces that self-report with external, mechanical checks: whether the test suite passes, whether the build runs clean from a cold clone, and whether the specific command tied to this task's definition of done exits zero. Our report on harness engineering makes the fuller argument, including why the process that checks the work must never be the one that did it.

Why Does an Overnight Loop Need a Hard Budget Cap?

Without a spend ceiling, an unattended agent keeps working and spending past the point where the work has any value. An automatically checked budget cap does for spend what a done gate does for completion, and it stops the loop even when nothing else does.

Agents do not get tired and have no built-in sense of when to stop trying. Given an open API budget and a task it cannot solve, a looping agent will keep retrying, generating tokens and running up a bill long after a person would have given up and asked for help. A hard budget cap enforced by the harness, as opposed to a polite request in the prompt, is what catches this specific failure. We set the cap per run and per task as well as per day, because one stuck task inside an otherwise healthy wave can burn the whole day's budget before anyone notices.

Which Actions Still Need a Human Approval Gate, Even Overnight?

Anything that leaves the sandbox and touches the outside world, such as a sent email, a merged pull request, a public post or a payment, needs a human approval gate no matter how good the loop's track record is. The loop can do the work unattended, but a person approves anything that goes out.

We hold one line without exception. An unattended loop can write code, run tests, open a draft and stage a change, but it never presses send. Emails, live deployments, merged pull requests, financial transactions and anything posted publicly under a real name all sit behind a human approval gate, however many clean waves the loop has already run. The reason is blast radius rather than distrust of any particular model: a bad commit in a branch takes a few minutes to revert, while a sent email or a live payment cannot be undone. We score every outward-facing action this way before it is allowed to run without review, and we treat a high score as necessary but not sufficient for letting the loop act alone.

What Is a Resilience Drill, and Why Run One Before Trusting a Loop Overnight?

A resilience drill deliberately breaks the environment before a real overnight run, for example by killing the process mid-task, revoking a credential or feeding the loop a malformed input, to confirm it fails safely instead of half-finishing silently. Without the drill, the first real failure is the test.

Before we trust a loop to run through a full night, we break it on purpose while someone is still watching. We kill the process mid-task and check that the state file still reflects reality on restart. We revoke a credential mid-run to confirm the agent stops cleanly instead of retrying over and over, and we feed it a deliberately malformed task to confirm it reports blocked instead of marking something done that never ran. A loop that has never been deliberately broken has an unknown failure mode, and you do not want to discover it at 3am with nobody watching.

Is Unattended AI Coding Actually Working in the Real World Right Now?

Adoption is high while trust is falling, and a harness is built to close that gap. Google's DORA research finds 90% of developers now use AI at work, while Stack Overflow's survey for the same year shows trust in AI accuracy dropped to 29%.

Right now the gap between how much AI coding tools are used and how much they are trusted is widening, and that gap is the argument for everything in this report. Google's 2025 DORA report, based on survey responses from nearly 5,000 technology professionals, found that 90 percent now use AI at work and more than 80 percent believe it increased their productivity, while 30 percent still report little or no trust in the code it generates. Stack Overflow's 2025 Developer Survey shows the same pattern. There, 84 percent of developers use or plan to use AI tools, up from 76 percent the year before, but trust in AI accuracy fell to 29 percent from 40 percent, and 66 percent report spending more time fixing AI output that was almost right but not quite.

The gap does not argue for avoiding unattended loops, only for never running one without the structure in this report. High adoption with falling trust means the industry already relies on output it does not fully believe, and a wave structure, a real done gate, a budget cap and an approval gate on anything outward-facing are there to make that survivable. We build all four into every long-running build in our workflow automation work. We also treat the instruction file the loop reads on every wave, covered in our report on writing instruction files agents actually follow, as the map that keeps a multi-hour run from drifting.