A live agent benchmark of independent action authorization for healthcare AI agents

Abstract Benchmarks measure what healthcare AI agents can do, not what they may be authorized to do. We built a live-agent benchmark holding task, model, prompt and tools fixed, varying only whether an independent authorization layer gated execution. 576 synthetic care-coordination episodes ran under four arms: deterministic rules, an unconstrained agent, a gateway-governed agent, and a gateway plus autonomy controller. Episodes carry hidden truth and a lossy observable projection, so the gateway can both wrongly authorize and wrongly block. A prespecified rule set blinded to arm adjudicated safety. Unsafe actions occurred in 0.7% of fully governed versus 0.4% of unconstrained episodes (+ 0.4 points, 95% CI -0.5 to + 1.2, P = 0.6875). Routine task success, escalation accuracy, and calibration were each significantly worse under full governance, which also required more time and human effort. These synthetic findings use rule-based adjudication, lack a human comparator, and do not establish clinical effectiveness.