Contextual Embedding Bias Predicts LLM Near-Synonym Confusion Where Isolated Geometry Fails: Evidence from Domain-Specific Medical and Cultural Heritage Corpora
Near-synonym confusion —- where a semantically similar but functionally distinct term is substituted for another –– constitutes a clinically important failure case of LLMs deployed in medical and high-stakes contexts. In retrieval- augmented generation safety literature, it is commonly assumed that the cosine similarity between isolated term embeddings predicts the probability of such a confusion. We directly test this assumption. We present NEARSYNONYMBENCH, a benchmark of 120 domain- specific near-synonym pairs from medical, Indian cultural heritage and emergency blood resource domains, annotated with confusion probes and stakes levels. We test four sentence transformer architectures and three LLM families under forced choice conditions. Our main result is dissociation: isolated embedding geometry accounts for no variance in LLM confusion behavior at all (r = −0.005, p = 0.958), while contextual embedding bias, or difference in the proximity of the query probe to each candidate term, constitutes a statistically significant predictor (ρ = 0.299, p < 0.001), accounting for 45.3 times more variance. This effect is especially pronounced for high stakes pairs (r = 0.312, p = 0.018).We additionally identify terminological evasion — accurate description without term commitment, as an underreported failure mode accounting for 28.3% of open-ended responses. NEARSYNONYMBENCH and all code are released publicly.
Authors
- Priyanka Asthana (ORCID: https://orcid.org/0009-0007-6709-0951)
Institutions
- Indian Army (IN)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-09-11
- DOI
- https://doi.org/10.5281/zenodo.22705033
- Primary Topic
- Biomedical Text Mining and Ontologies
- Type
- preprint