A working product is the spec agents were missing

Rewrites are a trap. You’ve heard it from every senior engineer you’ve worked with, and for decades they’ve been right. A rewrite costs a year, ships the old bugs plus new ones, and buys nothing a customer can see.

On September 10, Shopify announced it was doing one anyway. The company is moving its mobile apps off React Native, the cross-platform framework it adopted in 2020 so one team could write code for both iPhone and Android, and back to Swift and Kotlin, the languages each phone’s maker provides. One engineer built a proof of concept in a week. Then six engineers rebuilt the Shop app from scratch, all the way to production, in 12 weeks [reported by InfoQ, quoting Shopify’s Head of Mobile, Mustafa Ali].

Shopify says the results are good. Cold start is 23% faster on iPhone and 50% faster on Android. Crash-free sessions rose from 99.5% to 99.95%. Android release builds are 75% faster [reported, Shopify’s own numbers]. Ali’s reason is that coding agents “can now do enough of the implementation, translation, testing, and review work that it’s no longer the deciding factor it was in 2020.”

Agents are dependable when a shipping product exists to check them against, and they drift when it doesn’t. Your working product may be the most valuable input they can have.

That claim runs against something I’ve been tracking. Research on agents extending live codebases finds that success rates collapse from over 80% to 38% as the codebase grows [verified, three independent arXiv studies]. In most of the failures, the agent finds the right function to edit and then writes a wrong patch, because it can’t hold the whole system’s rules in its head. My working view was that agents wear down a system’s structure, because adding a feature is now so cheap that nobody stops to ask whether it fits.

A whole-app rebuild by agents that ended with fewer crashes doesn’t look like that curve. It looks like the opposite.

There are good reasons to doubt Shopify as a test. RedMonk, an industry analyst firm, points out that Shopify has “hundreds of mobile engineers and years of native expertise,” and that small teams still gain from sharing code across platforms. React Native now has independent governance under the React Foundation, so it isn’t going anywhere because one big user left. James Ide, co-founder of Expo, a company that builds tools on top of React Native, called the decision “more of a political decision than a technical one.” InfoQ adds that Shopify compared against the old React Native architecture, not the newer one it was about to adopt, and that native apps lose over-the-air updates, so every fix now waits on app store review [all reported]. And every performance number comes from Shopify.

All of that may be true. It still doesn’t explain the gap between the two results, because the two situations differ in a way you can check.

The studies with the collapsing success rates follow agents as they keep adding features to a codebase where nobody wrote down what the finished thing should do. Shopify’s agents worked from an app already running on customers’ phones. Every screen and behavior had an answer key. “Is the new version right?” meant comparing it with the old one. Ali describes the agents’ job as translation: carrying a feature from Android to iOS and back, with shared specs and tests keeping the two in line.

So the variable separating the two results might be less about how capable the agent is than about whether a reference exists to diff against.

Mathematics showed the same thing in September. Agents ran for about 88 hours on a famous open problem because a proof checker could grade their work without a human reading it. A rebuild has a checker too, and the checker is the app you already ship. Agents go as far as something can check their work.

This changes a decision you may be sitting on. Teams adopt abstractions like cross-platform frameworks, low-code layers, and shared component kits to save labor. Ali’s argument is that the labor-saving half of that trade has collapsed. If you own a working product and an abstraction you’re tired of paying for, you now hold something you didn’t before: the product itself, as the spec the rewrite always lacked.

I can’t tell you the rebuild will hold up. Shopify’s native apps are about to enter the regime the studies measure, where agents keep adding features with no answer key. If coherence erodes there on the same curve, Shopify got a one-time gain from having a reference, and it’s spending that gain now. That fits the reference reading as well as it fits any other. It’s a prediction, and a year from now someone can check it.

It also raises a question I don’t have an answer to. If a shipping product is what makes an agent reliable, what plays that role for a product that hasn’t shipped yet? Maybe the first working version, however rough, is the most valuable thing a team owns, and the sooner it exists, the sooner everything after it gets cheap to check.