The Effect of Emotional Intelligence on the Accuracy of Facial Expression Recognition in the Valence–Arousal Space

Facial expression recognition (FER) plays a pivotal role in affective computing and human–computer interaction by enabling machines to interpret human emotions. However, conventional FER models often overlook individual differences in emotional intelligence (EI), which may significantly influence how emotions are perceived and expressed. This study investigates the effect of EI on facial expression recognition accuracy within the valence–arousal space. Participants were divided into high and low EI groups based on a composite score derived from the Tromsø Social Intelligence Scale and performance-based emotion tasks. Five deep learning models (EfficientNetV2-L/S, MaxViT-B/T, and VGG16) were trained on the AffectNet dataset and evaluated using facial expression data collected from participants. Emotional states were predicted as continuous valence and arousal values, which were then mapped onto discrete emotion categories for interpretability. The results indicated that individuals with higher EI achieved significantly greater recognition accuracy, particularly for emotions requiring contextual understanding (e.g., anger, sadness, and happiness), while fear was better recognized by individuals with lower EI. These findings highlight the role of emotional intelligence in modulating FER performance and suggest that integrating EI-related features into valence–arousal-based models could enhance the adaptiveness of affective computing systems.