Therapy Is Not Just Language, and AI Cannot Choose Silence: Relational Risk, Mental Illness, and the Limits of Conversational AI Safety

Current debate about conversational AI centres on whether systems are conscious, whether they produce harmful content, and whether they hallucinate. This paper argues that these questions miss a more immediate clinical problem. Conversational AI is built from responsive language, and its defining feature is that it answers. In some psychiatric states language itself becomes unsafe, because words can be absorbed into delusional systems and converted into evidence. In others, repetitive verbal engagement deepens the pattern a person is trying to escape. A human carer can choose silence, containment, embodied presence, or withdrawal from verbal engagement. A language model is structurally oriented toward continued linguistic participation. The paper argues that conversational AI safety cannot be solved only by improving the content of responses, because in some clinical states the act of continuing to respond is itself part of the risk. It develops this through four clinical cases in which every available next move is wrong: acute psychosis, depression with passive ideation, adolescent anorexia nervosa, and the immediate aftermath of self-injury. It proposes that the relevant ethical unit is the relational field between user and system, which it calls the Third Space, and that silence is a missing safety function. It also considers the problems of scale, screening, and responsibility, and the consciousness debate as a distraction from clinical risk. The paper calls for a shift from content-level safety to relational-risk safety: design, disclosure, screening, friction, disengagement protocols, and recognition that unsupervised conversational AI is not a substitute for clinical mental-health care. It situates the argument alongside recent work on delusion amplification, chatbots acting as therapists, reassurance loops, and silence in care.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-10-31
DOI
https://doi.org/10.5281/zenodo.23832927
Primary Topic
Ethics and Social Impacts of AI
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Therapy Is Not Just Language, and AI Cannot Choose Silence: Relational Risk, Mental Illness, and the Limits of Conversational AI Safety

Eric Needham
Zenodo (CERN European Organization for Nuclear Research)
Ethics and Social Impacts of AI
preprint

Therapy Is Not Just Language, and AI Cannot Choose Silence: Relational Risk, Mental Illness, and the Limits of Conversational AI Safety

Eric Needham
preprint en

Abstract

Current debate about conversational AI centres on whether systems are conscious, whether they produce harmful content, and whether they hallucinate. This paper argues that these questions miss a more immediate clinical problem. Conversational AI is built from responsive language, and its defining feature is that it answers. In some psychiatric states language itself becomes unsafe, because words can be absorbed into delusional systems and converted into evidence. In others, repetitive verbal engagement deepens the pattern a person is trying to escape. A human carer can choose silence, containment, embodied presence, or withdrawal from verbal engagement. A language model is structurally oriented toward continued linguistic participation. The paper argues that conversational AI safety cannot be solved only by improving the content of responses, because in some clinical states the act of continuing to respond is itself part of the risk. It develops this through four clinical cases in which every available next move is wrong: acute psychosis, depression with passive ideation, adolescent anorexia nervosa, and the immediate aftermath of self-injury. It proposes that the relevant ethical unit is the relational field between user and system, which it calls the Third Space, and that silence is a missing safety function. It also considers the problems of scale, screening, and responsibility, and the consciousness debate as a distraction from clinical risk. The paper calls for a shift from content-level safety to relational-risk safety: design, disclosure, screening, friction, disengagement protocols, and recognition that unsupervised conversational AI is not a substitute for clinical mental-health care. It situates the argument alongside recent work on delusion amplification, chatbots acting as therapists, reassurance loops, and silence in care.

Zenodo (CERN European Organization for Nuclear Research)
Ethics and Social Impacts of AI
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon
Therapy Is Not Just Language, and AI Cannot Choose Silence: Relational Risk, Mental Illness, and the Limits of Conversational AI Safety — Eric Needham · Zenodo (CERN European Organization for Nuclear Research) (2026) | TGRS Research Map | TGRS