Earned Autonomy for AI Agents: A Pre-registered Error-Budget Test of Per-Action-Type Delegation

Which actions may an AI agent take without human sign-off? We propose an open rule: an agent earns autonomy on one action type after N consecutive verified successes and loses it at the first confirmed error; N is derived from an error budget fixed in advance per stakes level by adapting the unrestricted AOQL of continuous sampling plans, with a conditional bound on the extra errors delegation lets through. We test it in a pre-registered replay: three Claude 5.5 models on four public benchmarks (τ-bench airline, retail and banking; WorkBench), 14,266 episodes played (13,623 analysed), writes checked by exact comparison (98.7% coverage) and alerts reviewed on blind sheets by separate Claude instances. No configuration reached the pre-registered level of evidence required for delegation (verdict R4). Only one of twelve configurations had enough volume to be judged; its error rate, 8.6% (lower bound 4.1%), is established above the 0.76% threshold. Of its 79 errors, 48 are one plot end-date discrepancy reviewers decided both ways; lifting it throughout leaves 31, established by two errors, and lifting also four one-day window shifts would not. The other eleven had too little effective volume to conclude either way. Descriptively (no multiplicity correction), 20 of 40 model–environment–type combinations have an error rate established above their threshold (11 of them on banking); 13 show no observed error but too little volume. The rule was not refuted, but it was hardly put to the test, and its usefulness could not be judged. These rates count action errors confirmed by review against a single expected path per task; in a blind sample reviewed by the author, 3 of 10 confirmed alerts were lifted, too few to rule out false-alert shares that would overturn the judged excess. As the plan stated in advance, earning autonomy under such budgets takes far more verified actions than these benchmarks provide at our volumes.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-18
DOI
https://doi.org/10.5281/zenodo.24342386
Primary Topic
Ethics and Social Impacts of AI
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Earned Autonomy for AI Agents: A Pre-registered Error-Budget Test of Per-Action-Type Delegation

Otsoa Dubief
2 links
Zenodo (CERN European Organization for Nuclear Research)
Ethics and Social Impacts of AI
preprint

Earned Autonomy for AI Agents: A Pre-registered Error-Budget Test of Per-Action-Type Delegation

Otsoa Dubief
preprint en

Abstract

Which actions may an AI agent take without human sign-off? We propose an open rule: an agent earns autonomy on one action type after N consecutive verified successes and loses it at the first confirmed error; N is derived from an error budget fixed in advance per stakes level by adapting the unrestricted AOQL of continuous sampling plans, with a conditional bound on the extra errors delegation lets through. We test it in a pre-registered replay: three Claude 5.5 models on four public benchmarks (τ-bench airline, retail and banking; WorkBench), 14,266 episodes played (13,623 analysed), writes checked by exact comparison (98.7% coverage) and alerts reviewed on blind sheets by separate Claude instances. No configuration reached the pre-registered level of evidence required for delegation (verdict R4). Only one of twelve configurations had enough volume to be judged; its error rate, 8.6% (lower bound 4.1%), is established above the 0.76% threshold. Of its 79 errors, 48 are one plot end-date discrepancy reviewers decided both ways; lifting it throughout leaves 31, established by two errors, and lifting also four one-day window shifts would not. The other eleven had too little effective volume to conclude either way. Descriptively (no multiplicity correction), 20 of 40 model–environment–type combinations have an error rate established above their threshold (11 of them on banking); 13 show no observed error but too little volume. The rule was not refuted, but it was hardly put to the test, and its usefulness could not be judged. These rates count action errors confirmed by review against a single expected path per task; in a blind sample reviewed by the author, 3 of 10 confirmed alerts were lifted, too few to rule out false-alert shares that would overturn the judged excess. As the plan stated in advance, earning autonomy under such budgets takes far more verified actions than these benchmarks provide at our volumes.

Zenodo (CERN European Organization for Nuclear Research)
Ethics and Social Impacts of AI
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon