Contextual Embedding Bias Predicts LLM Near-Synonym Confusion Where Isolated Geometry Fails: Evidence from Domain-Specific Medical and Cultural Heritage Corpora

Near-synonym confusion —- where a semantically similar but functionally distinct term is substituted for another –– constitutes a clinically important failure case of LLMs deployed in medical and high-stakes contexts. In retrieval- augmented generation safety literature, it is commonly assumed that the cosine similarity between isolated term embeddings predicts the probability of such a confusion. We directly test this assumption. We present NEARSYNONYMBENCH, a benchmark of 120 domain- specific near-synonym pairs from medical, Indian cultural heritage and emergency blood resource domains, annotated with confusion probes and stakes levels. We test four sentence transformer architectures and three LLM families under forced choice conditions. Our main result is dissociation: isolated embedding geometry accounts for no variance in LLM confusion behavior at all (r = −0.005, p = 0.958), while contextual embedding bias, or difference in the proximity of the query probe to each candidate term, constitutes a statistically significant predictor (ρ = 0.299, p < 0.001), accounting for 45.3 times more variance. This effect is especially pronounced for high stakes pairs (r = 0.312, p = 0.018).We additionally identify terminological evasion — accurate description without term commitment, as an underreported failure mode accounting for 28.3% of open-ended responses. NEARSYNONYMBENCH and all code are released publicly.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-09-11
DOI
https://doi.org/10.5281/zenodo.22705033
Primary Topic
Biomedical Text Mining and Ontologies
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Contextual Embedding Bias Predicts LLM Near-Synonym Confusion Where Isolated Geometry Fails: Evidence from Domain-Specific Medical and Cultural Heritage Corpora

Priyanka Asthana
Zenodo (CERN European Organization for Nuclear Research)
Biomedical Text Mining and Ontologies
preprint

Contextual Embedding Bias Predicts LLM Near-Synonym Confusion Where Isolated Geometry Fails: Evidence from Domain-Specific Medical and Cultural Heritage Corpora

Priyanka Asthana
preprint en

Abstract

Near-synonym confusion —- where a semantically similar but functionally distinct term is substituted for another –– constitutes a clinically important failure case of LLMs deployed in medical and high-stakes contexts. In retrieval- augmented generation safety literature, it is commonly assumed that the cosine similarity between isolated term embeddings predicts the probability of such a confusion. We directly test this assumption. We present NEARSYNONYMBENCH, a benchmark of 120 domain- specific near-synonym pairs from medical, Indian cultural heritage and emergency blood resource domains, annotated with confusion probes and stakes levels. We test four sentence transformer architectures and three LLM families under forced choice conditions. Our main result is dissociation: isolated embedding geometry accounts for no variance in LLM confusion behavior at all (r = −0.005, p = 0.958), while contextual embedding bias, or difference in the proximity of the query probe to each candidate term, constitutes a statistically significant predictor (ρ = 0.299, p < 0.001), accounting for 45.3 times more variance. This effect is especially pronounced for high stakes pairs (r = 0.312, p = 0.018).We additionally identify terminological evasion — accurate description without term commitment, as an underreported failure mode accounting for 28.3% of open-ended responses. NEARSYNONYMBENCH and all code are released publicly.

Zenodo (CERN European Organization for Nuclear Research)
Indian Army (IN)
Sustainable cities and communities
Biomedical Text Mining and Ontologies
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.