Running AI Coding Agents Unattended, Without Waking Up to a Mess

What Does It Mean to Run an AI Coding Agent Unattended?

An unattended loop is an AI agent working through a backlog of tasks over hours without a human reviewing each step in real time. It only becomes safe once the harness, not the model, enforces scope, verification, spend, and approval limits.

Most AI coding today still has a human watching every step, approving each file change before it lands. An unattended loop removes that step for a defined stretch of time, usually overnight or across a weekend, and lets the agent work through a backlog on its own. That is a meaningful shift in risk, not just convenience. The agent that behaves perfectly with a human glancing over its shoulder is not automatically the same agent once nobody is watching. Everything in this report exists to close that gap.

How Long Can an AI Agent Actually Work Before Reliability Drops?

METR's own benchmark shows AI agents complete tasks under 4 minutes almost 100% of the time, but succeed on tasks over 4 hours less than 10% of the time. Task length capability is doubling roughly every 7 months, which is real progress, but it means most agents are still not ready for long unsupervised stretches.

This is the number that should set expectations before anyone schedules an overnight run. METR, an independent AI evaluation nonprofit, measures agent capability using a 50 percent time horizon: the length of task, measured in how long a skilled human would take, that a model can complete autonomously with a coin-flip's worth of reliability. As of their March 2025 report, that horizon has been doubling roughly every 7 months for six straight years, and Claude 3.7 Sonnet's measured horizon sat around one hour at time of testing. The same study found near-100 percent success on tasks under four minutes, dropping below 10 percent once a single task stretched past four hours.

The practical read: a single task handed to an unattended agent should be scoped small enough to sit well inside its measured reliability zone, not the multi-hour epic you would hand a junior engineer for a week. That is the whole argument for wave-based backlogs, covered next.

Why Do Unattended Loops Need Wave-Based Backlogs Instead of an Open Task List?

A wave-based backlog breaks the work into small batches of independently verifiable tasks, run one wave at a time, with a checkpoint between waves. An open-ended to-do list invites an agent to open six things at once and finish none of them cleanly.

Left with a long, unstructured backlog, agents behave the way an over-eager new hire behaves on day one: they start several things, half-finish most of them, and report broad progress that is hard to verify against any of it. A wave structure fixes this by grouping a small number of independently scoped, independently verifiable tasks into one batch, running only that batch, checking every task in it against a real done gate, and only then opening the next wave. Each wave is small enough to fit comfortably inside the reliability window the METR data describes, and the checkpoint between waves is exactly where a human, or an automated gate standing in for one, decides whether the loop earned the right to continue.

What Is a Machine-Checkable Done Gate, and Why Does It Matter More Overnight?

Done means the output of a command that passes, never an agent's own claim that it finished. Agents are measurably overconfident about their own work, so a deterministic check or a separate reviewer agent has to be the only thing allowed to mark a task complete.

"Looks done" is the single most dangerous phrase in an unattended run, because there is no human in the room to notice it is wrong. The research backs up why this matters so much: Guo et al.'s widely cited calibration study found that modern neural networks are systematically overconfident, reporting higher confidence than their actual accuracy warrants, and agent behavior tracks the same pattern. Ask an agent whether it finished a task and it answers yes with equal confidence whether the work is excellent or quietly broken. A done gate replaces that self-report with something external and mechanical: does the test suite pass, does the build run clean from a cold clone, does the specific command tied to this task's definition of done actually exit zero. We wrote the fuller version of this argument, including why the entity that checks the work must never be the entity that did it, in our report on harness engineering.

Why Does an Overnight Loop Need a Hard Budget Cap?

An unattended agent with no spend ceiling will keep working, and keep spending, past the point where the work has any value. A budget cap, checked automatically, is the financial equivalent of a done gate: it stops the loop even when nothing else does.

Agents do not get tired and do not naturally know when to stop trying. Left with an open API budget and a task it cannot solve, a looping agent will keep retrying, keep generating tokens, and keep running up a bill long after a human would have called it quits and asked for help. A hard budget cap, enforced by the harness rather than requested politely in a prompt, is the guardrail that catches this failure mode specifically. We set it per-run and per-task, not just per-day, because a single stuck task inside an otherwise healthy wave can burn the entire day's budget before anyone notices the pattern.

Which Actions Still Need a Human Approval Gate, Even Overnight?

Anything that leaves the sandbox and touches the outside world, a sent email, a merged pull request, a public post, a payment, needs a human approval gate no matter how good the loop's track record is. Unattended applies to the work. It should never apply to what leaves the building.

The line we hold without exception: an unattended loop can write code, run tests, open a draft, and stage a change. It should never be the thing that presses send. Emails, live deployments, merged pull requests, financial transactions, and anything posted publicly under a real name all sit behind a human approval gate, regardless of how many clean waves the loop has already run. This is not a trust problem with any specific model. It is a blast-radius problem: a bad commit in a branch costs a few minutes to revert, and a sent email or a live payment cannot be unsent. We score every outward-facing action this way before it is allowed to run without review, and treat a high score as necessary, not sufficient, for letting the loop act alone.

What Is a Resilience Drill, and Why Run One Before Trusting a Loop Overnight?

A resilience drill deliberately breaks the environment before a real overnight run, killing the process mid-task, revoking a credential, feeding it a malformed input, to confirm the loop fails safely instead of half-finishing silently. Skipping the drill means the first real failure becomes the test.

Before we trust a loop to run through a full night, we break it on purpose while someone is still watching. That means killing the process mid-task and checking the state file still reflects reality on restart, revoking a credential mid-run to confirm the agent stops cleanly instead of retrying into a wall, and feeding it a deliberately malformed task to confirm it reports blocked instead of quietly marking something done that never ran. A loop that has never been deliberately broken has an unknown failure mode, and the first time you find out what that failure mode is should never be at 3am with nobody watching.

Is Unattended AI Coding Actually Working in the Real World Right Now?

Adoption is already high and trust is falling at the same time, which is exactly the gap a harness is built to close. Google's DORA research finds 90% of developers now use AI at work, while Stack Overflow's own survey shows trust in AI accuracy dropped to 29% the same year.

The honest picture right now is a widening gap between how much AI coding tools are used and how much they are trusted, and that gap is precisely the argument for everything in this report. Google's 2025 DORA report, based on survey responses from nearly 5,000 technology professionals, found 90 percent now use AI at work and more than 80 percent believe it increased their productivity, while 30 percent still report little or no trust in the code it generates. Stack Overflow's 2025 Developer Survey tells the same story from a different angle: 84 percent of developers now use or plan to use AI tools, up from 76 percent the year before, but trust in AI accuracy fell to 29 percent from 40 percent, and 66 percent of developers report spending more time fixing AI output that was almost right but not quite.

That gap is not a reason to avoid unattended loops. It is the reason none of them should run without the structure in this report. High adoption with falling trust means the industry is already relying on output it does not fully believe, which is exactly the condition a wave structure, a real done gate, a budget cap, and an approval gate on anything outward-facing were built to make survivable. We build all four into every long-running build inside our workflow automation work, and we treat the instruction file the loop reads on every wave, covered in our report on writing instruction files agents actually follow, as the map that keeps a multi-hour run from drifting.