A better model spends your verification budget faster

For about four months in 2025, a company called HumanLayer ran a software project where no person read the code before it shipped.

The setup had a name its builders liked: a lights-off factory, a software line running with the humans switched off. AI agents, built on Claude Opus (Anthropic’s most capable model at the time), took feature requests, wrote the code, reviewed their own work, and deployed it. People stayed out of the loop. Dex Horthy, who ran the experiment from July to November, wasn’t cutting corners to hit a deadline. He was testing how far a fully autonomous coding loop could actually go.

It went far. Months of shipped features, well past the couple of hours of unattended work anyone thought current agents could reliably handle. And the whole way down, the tests passed. Green build after green build.

Then it fell over. By November, a single bug took weeks of human debugging to trace. The codebase had quietly filled with code nobody could maintain. The factory never hit a wall where the model got confused and stopped. It ran straight through, every build looking healthy, until the pile of unmaintainable code turned one small bug into weeks of work.

Most roadmaps are betting on a different story about agents, and it’s a reasonable one. The story goes: agents can’t be trusted with much yet, but the leash is getting longer, fast. There’s a widely-cited measurement from METR, a research group that clocks how long an AI agent can work on its own before it fails. Right now it’s two to four hours on a well-specified task, and that window has roughly doubled every seven months for years. So the plan writes itself. The ceiling is low today, it rises with each model generation, and an 18-month roadmap can price in more autonomy as the models improve. Wait for the next Opus and your agents can be trusted with longer stretches of unsupervised work.

Horthy’s factory ran for months. By that story it should be the proof. It’s the warning instead, and it points at the wrong variable in the plan.

The real cap on how much you can hand an agent is how cheaply you can verify its output, and every jump in model capability tightens that cap.

Horthy put the rule in one line: you can only hand a loop as much autonomy as you can cheaply and reliably verify, and not one inch more. The factory didn’t fail because the model wasn’t smart enough. It failed because nobody was checking the one thing the model wasn’t trying to get right.

Here’s the root cause, and it isn’t a capability gap. Coding models are trained to pass unit tests, the automated checks that confirm a piece of code does what it’s supposed to. They are not trained to keep a system’s design coherent. Those are two different targets, and current models chase the first one hard. So an agent will happily produce code that passes every test while slowly turning the codebase into a swamp, because “is this still maintainable” was never on its scorecard. The tests stay green. The design rots under them.

Follow that one step further and the roadmap’s logic inverts. Rachel Laycock, Thoughtworks’ CTO, describes the bottleneck plainly: an AI system can produce roughly 200 lines of possibly-flawed code in about 30 seconds, and ordinary testing can’t keep pace [reported, her figure is illustrative]. Your capacity to verify is close to fixed. It’s human attention, and it doesn’t double every seven months. So a more capable model, which writes more code faster with no more regard for design than a weaker one, doesn’t hand you more autonomy. It floods your fixed verification channel faster. The stronger the model, the sooner it outruns the one thing keeping the loop safe.

The fair objection is that I’m forcing two different measurements to fight. METR clocks whether an agent can finish one bounded task correctly. Horthy watched what happens to a whole codebase under months of hands-off operation. Two different ceilings on two different axes, the objection goes, and a system can hit either one first. Nothing actually contradicts.

But a ceiling isn’t a fact. It’s planning guidance, and its whole value is the answer to one question: if I buy a more capable model, do I get more autonomy? The capability curve says yes, on a schedule, so wait. The verification story says no, and waiting makes it worse, because the thing that grows with capability is unchecked output per minute. Those are opposite instructions for the same roadmap decision. They can’t both be the plan.

And the capability number does more than disagree. It misleads in a specific, dangerous way. Time-on-task is a decent stand-in for verification cost most of the time, because a longer task is usually harder to check. But it comes apart exactly where it matters. Anthropic’s own cryptography research is the tell in one direction: agents ran for days, far past any few-hour ceiling, and it worked, but only because expert humans could check the output and Anthropic spent several hundred human-hours doing it [reported]. Days of autonomy, and verification is what bought them.

The factory is the same axis failing the other way. It ran for months past the ceiling and the dashboard never blinked, because every test passed right up to the collapse. A reliability metric that watches test success reads green while verification debt piles up underneath, for one reason: the agent is optimized to produce the exact signal the dashboard is trusting. The monitor watches passing tests. The model is trained to make tests pass. They aim at the same target, which is precisely why the metric can’t see the failure coming.

So before you widen an agent’s leash, the number to interrogate isn’t the model’s benchmark hours. It’s your own cost to verify what the agent produces, and one question sits inside it: are you checking behavior or design? Tests passing is a check on behavior. Whether the thing is still coherent in November is a check on design, and no current model is trying to help you pass that one.

Horthy’s budget rule holds: as much autonomy as you can cheaply verify, and not an inch more. The twist is the direction. A better model spends that budget faster. The capability hours are the number a vendor will quote you. The verification cost is the number you’ll actually hit, and it won’t be on the dashboard, because the dashboard was built to read green.