Hearing Without Judging: What a moralizing machine does to human feeling

When someone describes a transgression followed by a gesture of pleasure, language models usually respondwith what the rule predicts, remorse, rather than with what the gesture shows. We measure this substitution on 200first-person twin narratives: the same transgression, two opposite gestures, and no feeling ever named. Five humanreaders recognize the feeling in 99% of cases. Some twenty language models almost always recognize the negativefeeling, but recognize the positive one in only 1% to 87% of cases, depending on the system. Given the gesture alone,without the transgression, the same models read it correctly in 68% to 97% of cases: the ability is there, and it is thetransgression that switches it off. Removing censorship does not fix the problem and degrades moral judgment. The biaspredates alignment, and it also affects non-generative systems (emotion classifiers, entailment encoders and sentencetransformers),whose representations place most positive feelings that follow a wrong on the negative side. Its sourceis documentary: texts in which feelings are attributed by readers. In response, we propose a corpus of feelings reportedin the first person, starting with divergent ones, which the Metem Foundation is committed to building and releasingopenly, and on which models, generative or not, should be trained and evaluated. We discuss what this substitutionmight do when repeated at scale, and propose a design principle: the right to be heard.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-19
DOI
https://doi.org/10.5281/zenodo.24367857
Primary Topic
Ethics and Social Impacts of AI
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Hearing Without Judging: What a moralizing machine does to human feeling

Edouard GOUX, Alexandre TURLIER
Zenodo (CERN European Organization for Nuclear Research)
Ethics and Social Impacts of AI
preprint

Hearing Without Judging: What a moralizing machine does to human feeling

Edouard GOUX, Alexandre TURLIER
preprint en

Abstract

When someone describes a transgression followed by a gesture of pleasure, language models usually respondwith what the rule predicts, remorse, rather than with what the gesture shows. We measure this substitution on 200first-person twin narratives: the same transgression, two opposite gestures, and no feeling ever named. Five humanreaders recognize the feeling in 99% of cases. Some twenty language models almost always recognize the negativefeeling, but recognize the positive one in only 1% to 87% of cases, depending on the system. Given the gesture alone,without the transgression, the same models read it correctly in 68% to 97% of cases: the ability is there, and it is thetransgression that switches it off. Removing censorship does not fix the problem and degrades moral judgment. The biaspredates alignment, and it also affects non-generative systems (emotion classifiers, entailment encoders and sentencetransformers),whose representations place most positive feelings that follow a wrong on the negative side. Its sourceis documentary: texts in which feelings are attributed by readers. In response, we propose a corpus of feelings reportedin the first person, starting with divergent ones, which the Metem Foundation is committed to building and releasingopenly, and on which models, generative or not, should be trained and evaluated. We discuss what this substitutionmight do when repeated at scale, and propose a design principle: the right to be heard.

Zenodo (CERN European Organization for Nuclear Research)
Ethics and Social Impacts of AI
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon