Facial Emotion Recognition Through a Smart Glasses Prototype: Improving Social Interaction for Visually Impaired Users Through Enhanced Deep Learning CBAM Architectures

This research focuses on creating a real-time facial emotion recognition system for smart glasses designed for visually impaired users. By adapting a contextually adaptive attention mechanism (CBAM) in a lightweight classification header with a pre-trained deep learning model, we obtain a model capable of successfully predicting emotions from facial features. The model will be complemented by a comprehensive preprocessing pipeline that includes face detection and alignment, standard normalization, and data augmentation for underrepresented classes. The model was trained on a merged benchmark dataset (FER24, RAF-DB, CK+) and evaluated across basic emotion classes: surprise, happiness, disgust, fear, sadness, neutral, and anger. Two models were compared: FaceNet–CBAM and EmoFormer–CBAM (a ViT-Base model enhanced with a 1D-CBAM attention module). EmoFormer–CBAM achieved 98% and ~72% test accuracy on new dataset (CleanFER25_RAF_CK) and AffectNet, respectively. In addition, a small set of external real-world images is used as a pilot qualitative evaluation to assess robustness under unconstrained conditions. A detailed analytical study of both models was performed to determine the impact of their structural components on overall performance using the available data. Based on the results, the most successful model under all conditions, EmoFormer–CBAM, was selected as the prototype for the smart glasses for the visually impaired. The necessary mechanisms for future deployment and implementation of the smart glasses prototype for the target users were also studied, in accordance with the ethical approvals previously obtained from Gazi University in Türkiye.