Facial Emotion Recognition with Cross-Dataset Generalization Capabilities

Abstract: Facial emotion recognition (FER) plays a vital role in human-computer interaction, mental health monitoring, and affective computing. Despite recent advancements, achieving generalization across datasets remains a significant challenge due to variations in pose, illumination, ethnicity, and annotation inconsistencies. This study investigates the cross-dataset generalization capabilities of deep learning-based FER models trained on AffectNet, RAF-DB, and FER2013, and evaluated on previously unseen datasets. The performance of convolutional neural networks (CNNs), vision transformers (ViTs), and domain adaptation techniques is systematically assessed. Experimental results indicate that a hybrid architecture combining ViT with domain-adversarial training achieves state-of-the-art generalization, with an average accuracy of 74.2% across multiple datasets. Statistical analyses further validate the robustness and real-world applicability of the proposed hybrid model. Keywords: facial emotion recognition, cross-dataset generalization, affective computing, vision transformers, CNNs, domain adaptation, deep learning, FER2013, AffectNet, RAF-DB