A Lightweight Framework for Facial Emotion Recognition: Leveraging Geometrical Priors with Spatial Features

Recognizing facial expressions (FER) continues to be a complex task due to the subtle similarities between certain emotion categories, such as disgust and anger, which have historically been challenging to differentiate. To counter this challenge, we present a novel multimodal deep learning architecture, that takes advantage of both geometric (landmark-based) and spatial pixel-wise (image-based) features for robust emotion classification. Our model integrates a Deep Convolutional Neural Network for image feature extraction with a fully connected network for facial landmarks, enhancing discrimination of similar emotions like disgust and anger. Achieving 67% accuracy on FER2013, 61% on AffectNet, and 94% on CK+, it demonstrates strong performance across diverse datasets, particularly in distinguishing subtle expressions. The fusion of modalities significantly improves robustness in facial expression recognition as a result our model is able to recognize confusing facial expressions (i.e. angry vs disgust). These findings demonstrate the durability of our multimodal strategy in dealing with the issues of inter-class similarity and variability in facial expression datasets, making it a promising solution for real-world FER applications.