Beyond Human Oversight: Three Forms of Independence for Safety Barriers in Clinical AI (Preprint)
UNSTRUCTURED Human oversight is widely invoked as protection against failures of artificial intelligence (AI) in clinical care, yet the presence of a human reviewer does not by itself establish an effective or independent safety barrier. A reviewer may rely on information derived from the same failed data pathway as the AI, form a judgment after being cognitively shaped by the AI recommendation, or work under workload, incentive, and authority constraints that make meaningful challenge difficult. Using bow-tie and barrier-management concepts, this Viewpoint distinguishes two analytically useful loss-of-control pathways: commission, in which unsafe AI-derived information or action is committed to care, and omission, in which a clinically necessary decision or action does not occur within a defined opportunity for intervention. We propose three forms of independence for human-dependent safety barriers: evidence independence, decision-formation independence, and organizational independence. Independence is not autonomy or organizational separation; it is the degree to which a barrier avoids the same information, cognitive, or operational dependencies that could cause the AI-enabled process to fail. The taxonomy is a conceptual proposal, not a validated scale. Human-factors mechanisms—including shared data pathways, anchoring, automation bias, vigilance decrement, workload, production pressure, and skill change—may degrade one or more forms of independence or the effectiveness of the barrier itself. We distinguish these three properties from five work-system conditions that may help establish and sustain them: fit to the work system, sustained competence, oversight capacity, calibrated reliance, and learning and correction. A discharge-summary example illustrates how a clinician can remain “in the loop” while contributing little independent barrier capacity and how redesign may improve independence without assuming that more review necessarily produces more safety. We conclude with candidate measures and falsifiable research questions. The central question for clinical AI safety is therefore not simply whether a human reviews an AI output, but independent of what, to what degree, with what net safety effect, and how is that independence maintained?
Authors
- Karthik Adapa (ORCID: https://orcid.org/0000-0002-3970-588X)
- Tarun Kumar Chauhan (ORCID: https://orcid.org/0009-0006-7530-8415)
- Myrna Martí (ORCID: https://orcid.org/0000-0002-7228-6795)
- Marcelo José D'Agostino (ORCID: https://orcid.org/0000-0002-6083-7713)
- Resham Sethi
- Padmini Vishwanath (ORCID: https://orcid.org/0009-0005-6500-4015)
- Mayank Garg
- Markos Mekasha Kelkay
- Mohammad Jarrahi
- Aura Bhattacharya
- Lukasz Mazur
Publication Details
- Published
- 2026-11-17
- DOI
- https://doi.org/10.2196/preprints.117193
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- preprint