AI can't check the options worth generating

Feed a strategy question to an AI today and it won’t hand you three options. It’ll hand you a hundred. That’s the pitch, and it’s a good one. A person can hold five or ten strategic directions in their head at once and compare them honestly. A model can generate and stress-test hundreds [reported]. Somewhere in that pile, the argument runs, sit the moves you’d never have reached on your own, including the politically radioactive ones your leadership team would have quietly buried before anyone said them out loud [reported].

More options, better strategy. That’s close to the entire case for pointing AI at strategy formation, and the volume is real. The trouble is what the volume is worth, and this week the research quietly answered a version of that question from the other end of the pipeline.

The finding is about a narrower thing: A/B tests. An A/B test is how a product team settles an argument with evidence. Show version A to some users, version B to others, measure which one wins. It’s slow and it costs traffic, so the obvious dream is to skip it: let an AI predict which version would win and never run the experiment. New work from Spotify’s engineering team and a companion paper (arXiv:2606.17165, “Statistical Foundations of LLM-based A/B Testing”) worked out exactly when you can trust that shortcut and when you can’t.

The answer has a sharp edge. The AI stand-in works when the thing you’re testing looks like things that have been tested before. It degrades as the new idea diverges from that history, and it degrades worst precisely where the idea is most novel [verified]. In the paper’s own line: A/B testing on an AI is correct by assumption; A/B testing on real humans is correct by design [verified].

Now put that next to the strategy pitch, because they’re measuring the same thing.

The property that makes an AI-generated option worth generating is the property that makes it uncheckable by AI. Both claims turn on one axis: distance from what’s been done before. The strategy case says value climbs with that distance. An option you’d have reached anyway adds nothing; an option beyond your reach is the whole point of running the generator. The validation research says reliability falls off with that same distance. The further an idea sits from your history, the less any AI can tell you whether it works. Value and verifiability run opposite directions along one line, and they cross.

Walk it slowly, because the second half is the part teams don’t see coming. Say you take the pitch and generate your hundred options. The honest objection to the whole system arrives right here: a human can’t evaluate a hundred options any faster than they could evaluate ten. So the natural fix is to reach for AI again. If the human is the bottleneck, have AI score the options too, rank the hundred, surface the winners before a person looks. That move is exactly what the surrogacy research studies, and its verdict is that the move is self-defeating for the novel options, the only ones the volume advantage was ever about. Confirming that the AI’s score is trustworthy for a genuinely new idea requires running the real test anyway, which defeats the point of skipping it [verified].

Worse: you can’t feel the failure from inside. The breakdown is structurally undetectable by the method doing the breaking [verified]. The AI returns a confident ranking of your hundred options whether or not its confidence means anything, and it means the least for the boldest ones.

The fair pushback is that these are different altitudes. A surrogate A/B test is about a headline or a button color; corporate strategy happens in a boardroom, and nobody validates a five-year direction with an automated experiment. True, and I’m not going to pretend otherwise. But the mechanism doesn’t care about altitude. It’s a fact about calibration, not about A/B tests. Any AI that scores a candidate by predicting its outcome against historical patterns is doing the same operation, and it goes blind in the same place: the part of the map it has no precedent for.

The strategy pitch has a built-in answer for rigor, and it’s worth noting that the answer isn’t the fix people think it is. The advice is to triangulate: reframe the question, run several methods, audit the three heaviest assumptions under any AI-generated strategy before you commit to it [reported]. Every one of those is interrogation. Interrogation isn’t validation. You can audit a novel option until its assumptions are the most examined in the building and still not know whether it works, because rigor at the frontier surfaces that the option is unvalidatable. It doesn’t make it validatable.

Here’s the part that should stop you. The strategy pitch’s proudest feature is that AI surfaces the options your org politics would have suppressed, the ones someone’s incentives or someone’s past decision made un-sayable. Surveys behind that pitch report real resistance to route around: roughly a third of employees admit to quietly sabotaging their company’s AI strategy, and a majority of CEOs report AI-related stress [reported]. But an option the room would have buried is, by definition, an option far from the room’s consensus. Which is to say: novel. The depoliticizing move and the novelty move are the same move. Every option AI gets credit for rescuing from suppression is an option AI cannot then check.

Political suppression was an ugly filter. Biased, self-serving, often wrong. But it was a filter, and it kept the untested-and-radical from reaching commitment on no evidence at all. The pitch is that AI removes that filter. The research says AI can’t supply a replacement at the frontier, and can’t tell you it hasn’t. You trade a biased-but-present human filter for an unbiased-but-absent AI one, in exactly the region where the options matter most because they’re most novel. The room used to bury its boldest ideas for the wrong reasons. Now it surfaces them for the right ones and has no trustworthy way to sort the visionary from the catastrophic, and no way to know it can’t.

So the practical work isn’t picking well from a rich set. It’s this. Sort your hundred options by distance from what you’ve already shipped. The near ones, where AI is just accelerating a pattern you’d have found eventually, you can let it help rank. The novel tail, the reason you ran the generator at all, you can only validate the old way: by testing on real people, which quietly caps how many you can ever actually commit to. The generator handed you a hundred doors. It can tell you which of the ones you’d have found anyway are worth opening, and nothing you can trust about the ones you wouldn’t. Those were the doors you paid for.