Empathy Without Assumptions: A Theory of Where Understanding Comes From Keeps an AI Coach's Replies Acceptable While Avoiding Unfounded Assumptions
Background. Language models used for coaching and emotional support can state feelings, causes or details about the user that the user never expressed. We treat such unfounded assumptions as a form of hallucination about the user. Instructions that forbid them may make replies less acceptable: the safest reply then repeats the person's words, and helping-skills research holds that repetition alone does not convey understanding. Objective. This exploratory pilot tested whether an AI coach instructed with a theory of where understanding comes from can avoid unfounded assumptions while keeping its replies acceptable as empathic. Methods. Three versions of the coach's empathy instructions (a one-sentence baseline, a reflection-only account of reflective listening and an evidence-grounded theory of empathy) ran on the same code. A general-purpose model given a one-line coaching prompt served as a fourth condition. Each condition replied five times to 24 fictional conversations (120 replies per condition). Two AI judges (large language models) rated every reply for acceptability and for unfounded assumptions. Four human raters rated 93 replies from an earlier run, to check how far the AI judges agreed with human judgement. Differences were analysed with the conversation as the unit, with 95% bootstrap intervals and a correction for 16 tests. Results. The evidence-grounded instructions produced acceptable replies far more often than the reflection-only instructions (71% against 38% for the primary AI judge; 53% against 14% for the secondary AI judge). They made unfounded assumptions about as rarely (4% each for the primary AI judge). They were also acceptable far more often than the general-purpose model (71% against 14%; 53% against 13%). Against the baseline, unfounded assumptions were lower (4% against 9%; 13% against 26%), but not significantly so. The AI judges agreed with the human raters' majority substantially (primary) and moderately (secondary) on unfounded assumptions. Conclusions. A short theory of where understanding comes from let an AI coach keep unfounded assumptions low without the loss of acceptability that reflection-only instructions caused. A confirmatory study will test the reduction relative to a minimal baseline.
Authors
- Anne Jamaluddin
- John Pinto (ORCID: https://orcid.org/0000-0002-4635-2226)
Institutions
- UCSI University (MY)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-18
- DOI
- https://doi.org/10.5281/zenodo.24347177
- Primary Topic
- Social Robot Interaction and HRI
- Type
- preprint