Multilingual Evaluation of Semantic Robustness and Cultural Adaptability in Transformer Models for Emotion Recognition

Multilingual language models achieve strong results in emotion classification; however, their robustness under culturally complex and linguistically diverse conditions remains limited. Standard evaluations rely on structured datasets that do not capture phenomena such as code-switching, idiomatic expressions, and indirect affective language, which are common in real-world communication. This study introduces a culturally oriented evaluation framework to analyze the behavior of mBERT, XLM-R, and a fine-tuned variant (XLM-R-FT) under non-standard linguistic conditions. The evaluation combines a multilingual dataset with a subset of manually annotated samples incorporating regional variation in Spanish, Portuguese, and Italian. Results show consistent performance degradation on culturally marked inputs, with reductions of up to 10 F1 points. Fine-tuning improves stability but does not eliminate sensitivity to implicit and idiomatic expressions. Interpretability analysis using LIME and SHAP indicates that predictions are often driven by lexically salient tokens, leading to misinterpretations. The proposed framework enables evaluation beyond standard benchmarks and highlights the need to incorporate cultural and linguistic variability in multilingual emotion analysis.

Authors

Institutions

Publication Details

Journal
Cognitive Computation
Published
2026-06-30
DOI
https://doi.org/10.1007/s12559-026-10621-7
Primary Topic
Emotion and Mood Recognition
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Multilingual Evaluation of Semantic Robustness and Cultural Adaptability in Transformer Models for Emotion Recognition

Aracely Mera-Navarrete, William Villegas-Ch, Rommel Gutiérrez, Rodrigo Guevara-Reyes
Cognitive Computation
Emotion and Mood Recognition
article

Multilingual Evaluation of Semantic Robustness and Cultural Adaptability in Transformer Models for Emotion Recognition

Aracely Mera-Navarrete, William Villegas-Ch, Rommel Gutiérrez, Rodrigo Guevara-Reyes
article en

Abstract

Multilingual language models achieve strong results in emotion classification; however, their robustness under culturally complex and linguistically diverse conditions remains limited. Standard evaluations rely on structured datasets that do not capture phenomena such as code-switching, idiomatic expressions, and indirect affective language, which are common in real-world communication. This study introduces a culturally oriented evaluation framework to analyze the behavior of mBERT, XLM-R, and a fine-tuned variant (XLM-R-FT) under non-standard linguistic conditions. The evaluation combines a multilingual dataset with a subset of manually annotated samples incorporating regional variation in Spanish, Portuguese, and Italian. Results show consistent performance degradation on culturally marked inputs, with reductions of up to 10 F1 points. Fine-tuning improves stability but does not eliminate sensitivity to implicit and idiomatic expressions. Interpretability analysis using LIME and SHAP indicates that predictions are often driven by lexically salient tokens, leading to misinterpretations. The proposed framework enables evaluation beyond standard benchmarks and highlights the need to incorporate cultural and linguistic variability in multilingual emotion analysis.

Cognitive ComputationVol. 18(1)
Universidad Estatal de Milagro (EC), Universidad de Las Américas (EC), Universidad Internacional del Ecuador (EC), Universidad Andina Simón Bolívar (BO)
Quality Education
Openalex Percentile: Top 6%
Emotion and Mood Recognition
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.