EmoVisioNet: A hybrid network unifying lightweight CNN and attention-based vision model for facial emotion detection

Facial emotion detection has witnessed a surge in demand across numerous applications, including human-computer interaction, healthcare, and security. Accurate expression recognition is crucial for improving human-computer interactions and understanding human behavior. Existing facial emotion detection models face challenges in achieving both high accuracy and real-time processing due to complex architectures. Our goal is to create an efficient yet accurate solution that can work on resource-constrained devices. To address the challenge of accurately recognizing emotions from facial expressions, we propose a novel hybrid approach that combines the strengths of pretrained Lightweight Convolutional Neural Networks (CNN), and Attention-based Vision Models. The pretrained Lightweight CNN serves as a feature extractor, efficiently capturing facial features, while the attention model refines the feature representation to focus on crucial regions of the face associated with different expressions. This enables our model to achieve state-of-the-art (SOTA) accuracy with reduced computational requirements. The proposed model, EmoVisioNet, achieves superior performance across multiple datasets, attaining 99.97 % accuracy on CK+, 96.23 % on RAF-DB, 93.88 % on FER2013, and 96.91 % on FERPlus. The obtained results surpass the current state-of-the-art in this field, demonstrating the EmoVisioNet’s superior performance in facial expression recognition. • We propose EmoVisioNet: a hybrid CNN-attention model for emotion recognition. • EmoVisioNet uses a stacked ensemble of MobileNet and self-attention-based encoders. • We introduce a patch extraction block to enhance local facial features. • EmoVisioNet achieves 99.97 % accuracy on CK+, and 96.23 % on RAF-DB benchmark datasets. • EmoVisioNet is designed for real-time performance on resource-constrained devices.