Disclosure scaled to stakes goes silent when the stakes are invisible

You’re setting the disclosure policy for an AI feature, and the reasonable line writes itself. Flag the AI when the stakes are high. Skip the label when they’re low. A medical summary or a contract clause gets a clear “generated by AI.” A rough first-draft email doesn’t need the ceremony. Disclose in proportion to what’s at risk, and you spend your users’ attention where it earns its keep.

It’s a good rule. It respects people’s time, it matches the effort to the consequences, and most transparency guidance you’ll read lands somewhere near it. The same logic applies to showing an AI’s reasoning: doing it has a real cost. When an AI explains itself at every step, people decide slower, and research finds they start rubber-stamping recommendations they never actually read [reported]. When they do read the reasoning, they get better at telling good output from bad [verified]. So you save the heavy disclosure for the decisions that can hurt someone, and let the rest stay quiet.

Now look at what a new piece of evidence does to that rule. Martin Fowler, a widely read software writer, recently pulled together the controlled studies on whether people can tell AI-written text from human-written text [reported, Sep 1 2026]. They can’t. Across studies, humans identify AI-generated text at 50 to 57 percent accuracy [verified]. The floor there is a coin flip. On text people wrote themselves they do a little better, 64 percent [verified]. A Stanford-affiliated study had reviewers read academic abstracts and mark which were machine-written; they misfiled the AI ones as human about 62 percent of the time [reported]. The old tells, the stiff phrasing and the repetitive structure, don’t survive contact with current models.

Then the part that matters most. People’s confidence in their own judgment has no relationship to whether they’re right [reported]. The reader who’s sure it’s human is no more accurate than the one who’s guessing.

Sit that next to the rule and the problem is structural, not a detail. Scaling disclosure to the stakes only works if someone can see the stakes, and whether content is AI-made is a stakes-raising fact nobody in the loop can see. The rule quietly assumes the high-stakes condition announces itself. When a user is about to authorize a payment, it does. The amount is right there on the screen. But “this was written by an AI” changes how much you should trust what you’re reading, and it arrives with no such signal. The user can’t detect it. Worse, the user’s sense that they have detected it is worthless, because confidence and accuracy have come unhooked.

Watch what a stakes-scaled system does with a trigger it can’t perceive. No signal of raised stakes reads as low stakes. Low stakes means stay quiet. So the rule defaults to silence in precisely the situation it was built to catch, and it does it calmly, with nothing on screen to suggest anything was held back. “Disclose when it matters” collapses into “disclose when the user already suspects,” which is the one moment disclosure was least needed.

There’s a fair defense of the rule, and it’s worth stating at full strength. Authorship, someone will say, is just a special category. Where a text came from is a different question from what a decision costs, so of course you make provenance always-on and scale everything else. Carve authorship out, and the rule survives intact for reasoning, for citations, for the ordinary transparency work it was written for.

That carve-out is too tidy, and the reason is what makes this worth your attention. Authorship is the first documented case of something the rule never accounted for: a condition that raises the stakes of trusting content while staying invisible to the person who’d act on it. Once you see the shape, you start finding it in other places.

Take silent product changes. Earlier this year, users of Anthropic’s AI coding tool were convinced the model had gotten dumber. The post-mortem found the cause wasn’t the model at all. Three product-layer tweaks had shipped quietly: a lowered reasoning setting, a caching bug, and a cap on how much the model would say [verified]. To the user, a routine performance change and a real loss of capability looked identical. The thing that should have changed how much they trusted the tool was invisible to them. Same shape as authorship.

Take the habit we lean on hardest, having users double-check AI output. That’s the fallback everywhere. The model drafts, the human verifies. Workday, the HR-software company, found in its own research that distrustful users re-check everything the AI produces, which erases the time the feature was supposed to save [verified]. But re-checking only works if you can tell a right answer from a wrong one. If your confidence in that check is as uncalibrated as your confidence in spotting AI text, the verification is theater. You feel careful and you’re guessing.

I want to be careful about how far this goes. The detection numbers are solid. The claim that miscalibrated confidence spreads from spotting AI text to every judgment about when to scrutinize output is my read, not a measured result, and it deserves testing before anyone builds on it. What the evidence does establish is narrower and still sharp: in at least one important case, people can’t perceive a condition that should change how much they trust what’s in front of them, and they can’t perceive that they can’t.

So the rule doesn’t need to be scrapped. For triggers a person can actually see (an irreversible action, a payment you can’t claw back), scaling disclosure to the stakes is still right, and the cost of over-disclosing is still real. What the rule needs is the clause it never had: where the stakes-raising condition is invisible to the user, disclosure can’t be scaled to it and has to default on.

Which turns the useful question away from your disclosure copy and toward your trigger. What condition, exactly, flips disclosure on in your product? Then the harder one. Can the user, or your own system at runtime, actually perceive that condition? AI authorship is the case where the answer is now provably no. The open question, the one worth chasing, is how many other blind triggers your product already has, each one a spot where a sensible rule stays quiet at exactly the wrong time.