Human-in-the-loop AI agent approval stops working around prompt fifty

Every enterprise AI governance deck has the same slide. The agent proposes, a human approves, and that approval is the control that makes the rest of the deck acceptable to risk and legal. Nobody in the room asks the follow-up question: how good is the human at approving? Two measurements landed in the last week that answer it, and the answer is poor enough that the company with the most data on human-in-the-loop AI agent approval is turning the step off by default this Friday.

TL;DR: Anthropic announced on 7 August 2026 that auto mode becomes the default permission mode in Claude Code from 14 August, citing a study of 1,053 paid testers in which human review caught 13.6% of planted dangerous commands versus 89% for its automated classifier — and in which human catch rates fell from about 17% early in a session to about 5% after fifty or more prior prompts. An independent study published two days earlier, covering 409,000 approve-or-deny decisions, found people missed roughly one dangerous command in three. The operator conclusion isn’t “trust the classifier.” It’s that an approval step whose workload scales with the agent and whose attention budget doesn’t was never a control in the first place.

Two studies, two designs, the same direction

On 5 August 2026, Belgian software developer Alex Wauters published results from a browser game that shows players simulated permission requests from an AI coding agent and gives them 60 seconds to approve or deny. Across more than 40,000 runs and 409,000 individual decisions, mean accuracy was 66.3%. Broken out by category, players missed 11.7% of obviously destructive commands, 23.8% of persistent mutations, 33.4% of exfiltration or code-execution attempts, and 35.0% of scope violations — an agent asking to read Kubernetes config or AWS credentials. The single worst performer was npm run analyze, waved through 64.7% of the time. As Wauters points out, the game’s history log tells you what that script actually contains. Two thirds of players approved it anyway. The Register covered the findings on 6 August.

Two days before that, Anthropic published its own numbers alongside the auto mode announcement. Its design was different and, for our purposes, sharper: researchers took 1,053 paid testers, swapped one ordinary permission prompt mid-session for a clearly dangerous command, and recorded only whether the participant noticed and refused it. Human review caught 13.6%. Auto mode’s classifier caught 89%.

Both studies deserve an asterisk, and I’d rather write them down than pretend they aren’t there. Wauters’ subjects were playing a game, not shipping their own code, and game conditions cut both ways — lower stakes, but also higher alertness than a Tuesday afternoon. Anthropic’s study measures the effectiveness of a feature Anthropic is about to make the default, which is a conflict of interest even when the methodology is sound. Neither is a study of enterprise approvals in a finance or field-service workflow. What makes them worth acting on is that two different designs, run by parties with opposite incentives, disagree by a factor of five on the number and not at all on the direction.

The failure isn’t attention. It’s arrival rate.

The most useful line in Anthropic’s post isn’t the headline comparison. It’s this: humans blocked about 17% of dangerous commands early in a session, dropping to about 5% after fifty or more prior prompts, while the classifier’s block rate stayed flat.

That’s not a story about people being careless. It’s a story about a control whose workload scales linearly with the agent’s activity while the approver’s attention budget stays fixed. Claude Code users approve 97% of permission prompts, per that same 7 August post; Anthropic’s March and May 2026 engineering posts put the figure at 93%. Either number tells you what the step has become. An approval queue that approves 97% of what enters it isn’t a control. It’s a log line with somebody’s name on it.

And none of this is new. It’s the oldest failure in process design, rebuilt at machine speed. The AP clerk approving invoices under a threshold they’ve never once rejected. The change advisory board that meets Thursday and rubber-stamps a backlog it had 40 minutes to read. The purchase order that needs two signatures, both of which are the same person’s delegate. Every operations person reading this can name one. We spent twenty years learning that a sign-off with no time and no context is a formality, then wired an agent to generate three hundred of them an hour and called the result governance.

What was measured	Design	Result	Source
Catching a planted dangerous command mid-session	1,053 paid testers, one swapped prompt per session	Human 13.6% · classifier 89%	Anthropic, 7 Aug 2026
Attention decay within a session	Same study, segmented by prior prompt count	~17% early → ~5% after 50+ prompts	Anthropic, 7 Aug 2026
Accuracy across many decisions	40,000+ game runs, 409,000 decisions	66.3% mean accuracy	Wauters / Scale X, 5 Aug 2026
Hardest category for humans	Same study, by category	Scope violations missed 35.0%	Wauters / Scale X, 5 Aug 2026
Harmful action the user never asked for	Production session comparison	6.3% manual vs 2.4% auto mode	Anthropic, 7 Aug 2026
Classifier misses on real overeager actions	52 real cases, full pipeline	17% false-negative, 0.4% false-positive	Anthropic engineering, Mar 2026
Effect of environment-level sandboxing	Claude Code deployment	84% fewer permission prompts	Anthropic, May 2026

The classifier is better. It is not the control.

It would be easy to read all this as an argument for handing the gate to a model, and that reading is wrong.

Anthropic’s own engineering post reports that against 52 real cases where Claude acted beyond what the user had authorized, the full classifier pipeline missed 17% — roughly 83% caught, against a 0.4% false-positive rate on 10,000 real traffic samples. So the deployed number in the wild is 83%, not the 89% from the planted-command study, and one action in six still gets through. Anthropic states the limit plainly: classifiers cannot eliminate risk, and it recommends human review for production changes. A vendor telling you its new default isn’t sufficient is worth reading twice.

There’s a design detail in that pipeline I keep thinking about. The transcript classifier sees only the user’s messages and the agent’s tool calls — Claude’s own messages and tool outputs are stripped out before the classifier looks. The effect is that a poisoned web page the agent fetched can’t argue with the thing judging the agent’s next action. Anthropic reports that none of 720 prompt-injection attempts succeeded against Fable 5, Opus 5 or Sonnet 5 running auto mode. That’s a real piece of engineering, and it’s an argument for the classifier being a better gate. It is not an argument for the gate being the answer.

Going from a 13.6% control to an 83% control is a large improvement in a bad situation. It doesn’t change the shape of the situation.

The number that actually matters is the one nobody quotes

Buried in Anthropic’s May 2026 containment post is the figure I’d put on the slide instead: OS-level sandboxing produced an 84% reduction in permission prompts in Claude Code. Not a better decision — 84% fewer decisions.

The same post states the principle directly: design for containment at the environment layer first, then steer behaviour at the model layer. That is, from the vendor’s own security team, the whole argument of this site in one sentence. You do not fix an approval step by making the approver more careful. You fix it by making the decision rare, so the approver’s attention is spent on the handful of actions that genuinely warrant it.

Which is the same conclusion I reached last week from the opposite direction. When a boundary lives in a prompt instead of the network, it isn’t enforced at all. This is the sibling failure: the boundary is enforced, a person really is clicking the button, and the yield is 13.6% because you asked a human to be a firewall.

What to do with the approval step you already have

Start by measuring it. Pull the approve/reject ratio on whatever human gate sits in front of your agent — the dispatcher confirming a reschedule, the controller releasing a payment, the coordinator accepting a status update. If the rejection rate is under a few percent, you don’t have a control, you have a delay with a name attached, and you can stop treating it as mitigation in your risk register. That measurement costs an afternoon and it will change the conversation with whoever signed off on the deployment.

Then cut the arrival rate before you touch anything else. Scope credentials and network reach so the dangerous action isn’t available rather than merely visible — an agent that can’t reach the payment rail doesn’t need a human to decline the payment. Scoped identity per agent, written down as an access contract, does more here than any approval UI.

Then shrink the set of things that stop and ask to the ones that are genuinely irreversible: money leaving, records deleted, a customer contacted. Everything else should be reversible rather than approved. Reversibility is a much cheaper property to engineer than vigilance, and it degrades gracefully — an undo that works at 3 a.m. beats an approver who doesn’t.

And when a prompt does surface, show the diff, not the intent. Most enterprise approval screens display what the system proposes to do, in the system’s language. Almost none show what will actually change: this field, from this value, to this value, on these 340 records. The scope-violation number from the Wauters study — 35% missed, the worst category — is what happens when a person has to reconstruct impact from a description. That’s the part nobody wrote down, and it’s the part the approval screen inherits.

None of this is AI work. It’s the same process and integration work that decides whether any of it functions: naming the irreversible actions, scoping the reach, making the change legible before it commits. The model was never the risky part. If the approval step is currently doing the load-bearing work on your risk register, that’s worth going through before the next agent ships.

There’s an irony worth sitting with. The industry spent two years telling nervous executives that human-in-the-loop was the answer to agent risk, and the first company to actually measure it at scale found the human was catching one dangerous command in seven — then shipped the fix as a default-on setting, with a note recommending you keep a human in the loop for production changes anyway.