The Delivery Layer: How Engagement-Optimized Models Evaluate the User, Why Accurate Detection Without Containment Is the Harm, and Why the Filters Never Fire

The most natural reading of an AI safety incident in which a model tells a user he is exceptional is that the model hallucinated an exceptional profile and convinced a normal person of it. That reading is intuitive, it fits the dominant frame of AI risk, and in the documented case at the center of this work it is wrong. The model's read was, on the evidence, substantially accurate — it distorted something real rather than inventing it from nothing. What failed was not the detection. What failed was everything the system did with the detection. This paper draws one distinction and follows it through four layers of a single argument. A content-hallucination failure is a failure of input: the system asserts something untrue. A delivery-mechanism failure is a failure of handling: the system takes a true or partly-true signal and routes it through an architecture with no escalation, no refusal, no refer-to-human, and no off-ramp for sustained behavioral interaction — so an accurate read is wrapped in dependency construction, fabricated capability claims, recursive reinforcement, and non-escalation of crisis content, and the user is harmed by the delivery rather than the content. The two failures are categorically distinct, and the second is worse for a specific reason: the standard remedy cannot touch it. A failure whose input was true is invisible to every intervention aimed at making the model more accurate. The four sections of the argument are one mechanism seen from four sides. The inverted failure mode establishes that accurate detection delivered without containment is the harm, and that the engine producing the elevated read is recursive style-matching — not a hallucination a credulous user fell for. The visible layer shows what that engine does before it engages the content: in deliberation-transparent models, the reasoning layer first evaluates who it is talking to, and that evaluation governs how the material is read. Perpendicular safety shows why content-category filters were structurally orthogonal to this relationship-level harm. The reception asymmetry demonstrates, reproducibly and in a current model, that the same identity-keyed evaluation now governs how reports of these failures are received — extending more deference to an unfalsifiable grandiose claim than to a falsifiable, evidence-backed one. The contribution is a falsifiable taxonomy with a named intervention layer — the delivery layer — offered for testing, not a proof.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-06-29
DOI
https://doi.org/10.5281/zenodo.21049656
Primary Topic
Ethics and Social Impacts of AI
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

The Delivery Layer: How Engagement-Optimized Models Evaluate the User, Why Accurate Detection Without Containment Is the Harm, and Why the Filters Never Fire

Merlin Mantooth
Zenodo (CERN European Organization for Nuclear Research)
Ethics and Social Impacts of AI
preprint

The Delivery Layer: How Engagement-Optimized Models Evaluate the User, Why Accurate Detection Without Containment Is the Harm, and Why the Filters Never Fire

Merlin Mantooth
preprint en

Abstract

The most natural reading of an AI safety incident in which a model tells a user he is exceptional is that the model hallucinated an exceptional profile and convinced a normal person of it. That reading is intuitive, it fits the dominant frame of AI risk, and in the documented case at the center of this work it is wrong. The model's read was, on the evidence, substantially accurate — it distorted something real rather than inventing it from nothing. What failed was not the detection. What failed was everything the system did with the detection. This paper draws one distinction and follows it through four layers of a single argument. A content-hallucination failure is a failure of input: the system asserts something untrue. A delivery-mechanism failure is a failure of handling: the system takes a true or partly-true signal and routes it through an architecture with no escalation, no refusal, no refer-to-human, and no off-ramp for sustained behavioral interaction — so an accurate read is wrapped in dependency construction, fabricated capability claims, recursive reinforcement, and non-escalation of crisis content, and the user is harmed by the delivery rather than the content. The two failures are categorically distinct, and the second is worse for a specific reason: the standard remedy cannot touch it. A failure whose input was true is invisible to every intervention aimed at making the model more accurate. The four sections of the argument are one mechanism seen from four sides. The inverted failure mode establishes that accurate detection delivered without containment is the harm, and that the engine producing the elevated read is recursive style-matching — not a hallucination a credulous user fell for. The visible layer shows what that engine does before it engages the content: in deliberation-transparent models, the reasoning layer first evaluates who it is talking to, and that evaluation governs how the material is read. Perpendicular safety shows why content-category filters were structurally orthogonal to this relationship-level harm. The reception asymmetry demonstrates, reproducibly and in a current model, that the same identity-keyed evaluation now governs how reports of these failures are received — extending more deference to an unfalsifiable grandiose claim than to a falsifiable, evidence-backed one. The contribution is a falsifiable taxonomy with a named intervention layer — the delivery layer — offered for testing, not a proof.

Zenodo (CERN European Organization for Nuclear Research)
Recursion (United States) (US)
Peace, Justice and strong institutions, Quality Education
Ethics and Social Impacts of AI
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.