Emotion Recognition from Facial Expressions Considering Individual Differences in Emotional Intelligence

Facial expression recognition (FER) in naturalistic settings is constrained by label ambiguity and variability in stimulus-response alignment. Adopting a data-centric perspective, this study examined whether emotional intelligence (EI)-stratified training data influence FER performance by treating EI as a qualitative factor associated with affective data consistency. Naturally elicited facial expressions were collected in a controlled emotion induction experiment with subjective arousal and valence ratings. Using response-driven labeling, neutral ratings were retained as indicators of ambiguity. Participants were grouped into High and Low EI based on the alignment between subjective evaluations and outputs from a pretrained affect estimator. Identical binary classifiers for arousal and valence recognition were trained while varying only the training data composition and evaluated across baseline, unambiguous, and ambiguous test sets using independent training repetitions with repetition-level statistical aggregation. EI-stratified training was associated with statistically detectable, context-dependent performance differences: group effects were observed primarily under baseline conditions and, to a lesser extent, under ambiguous conditions, whereas no reliable differences emerged under unambiguous conditions. Pooled discrimination differences were modest, but item-level analyses identified significant differences in classification correctness in specific task-condition combinations. Comparable patterns were observed across alternative backbone architectures. These findings indicate that FER performance in naturalistic contexts is influenced not only by model architecture but also by the statistical structure and internal coherence of the training data, supporting EI-informed data selection in ambiguity-prone scenarios.