Why Most AI Incidents Are Evidence Failures, Not Model Failures

Public discourse on AI risk continues to frame incidents primarily as technical failures: model bias, hallucination, or misconfiguration. This article advances a different interpretation grounded in governance practice. Drawing on patterns observable in the OECD AI Incidents Monitor, it argues that many AI incidents escalate not because models fail, but because institutions cannot reconstruct what AI systems said, when they said it, and how those representations were framed at the moment of reliance. The article does not assess model accuracy, internal system design, or causality. Instead, it examines AI incidents as post-event accountability failures driven by missing or non-inspectable evidence. Through sector-agnostic walkthroughs spanning finance, healthcare, and public administration, it demonstrates a recurring governance failure mode: once scrutiny occurs, the absence of contemporaneous, interaction-specific records converts uncertainty into institutional exposure regardless of technical intent or system quality. The paper reframes AI incident management as an evidentiary control problem rather than a model optimization problem. It concludes that, in non-deterministic systems deployed as external representation channels, accountability depends less on improving prediction accuracy than on preserving inspectable records of AI-mediated representations at the point of human reliance.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-01-09
DOI
https://doi.org/10.5281/zenodo.18196750
Primary Topic
Ethics and Social Impacts of AI
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Why Most AI Incidents Are Evidence Failures, Not Model Failures

Tim de Rosen
Zenodo (CERN European Organization for Nuclear Research)
Ethics and Social Impacts of AI
article

Why Most AI Incidents Are Evidence Failures, Not Model Failures

Tim de Rosen
article en

Abstract

Public discourse on AI risk continues to frame incidents primarily as technical failures: model bias, hallucination, or misconfiguration. This article advances a different interpretation grounded in governance practice. Drawing on patterns observable in the OECD AI Incidents Monitor, it argues that many AI incidents escalate not because models fail, but because institutions cannot reconstruct what AI systems said, when they said it, and how those representations were framed at the moment of reliance. The article does not assess model accuracy, internal system design, or causality. Instead, it examines AI incidents as post-event accountability failures driven by missing or non-inspectable evidence. Through sector-agnostic walkthroughs spanning finance, healthcare, and public administration, it demonstrates a recurring governance failure mode: once scrutiny occurs, the absence of contemporaneous, interaction-specific records converts uncertainty into institutional exposure regardless of technical intent or system quality. The paper reframes AI incident management as an evidentiary control problem rather than a model optimization problem. It concludes that, in non-deterministic systems deployed as external representation channels, accountability depends less on improving prediction accuracy than on preserving inspectable records of AI-mediated representations at the point of human reliance.

Zenodo (CERN European Organization for Nuclear Research)
Peace, Justice and strong institutions
Openalex Percentile: Top 93%
Ethics and Social Impacts of AI
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.