Context Attentive Dual Stream Emotion Aware Swin Transformer for Academic Emotion Recognition

Abstract Enhancing academic engagement and adaptive learning systems requires an understanding of students’ emotions in the classroom. Conventional facial expression recognition techniques frequently fall short in educational environments because of issues such inadequate lighting, occlusion, changing head positions, and the requirement to record contextual data outside of the face. With three significant innovations, this study suggests a unique framework for robust academic emotion recognition called Context Attentive Cascaded Preprocessing with Dual Stream Emotion Aware Swin Transformer (CADE-Swin) to overcome these constraints. To preserve both fine-grained facial expressions and broader behavioral context, a Context Attentive Multi-Task Cascaded Convolutional Neural Network (CA-MTCNN) preprocessing is first constructed to jointly extract aligned face crops and face-context regions. Second, by combining emotion-aware attention heads, the Dual Stream Emotion Aware Swin Transformer (DE-Swin) architecture prioritizes salient facial and action unit regions while processing face-only and context crops simultaneously. Third, sequence-level reasoning for continuous engagement analysis is made possible by the extension of the DE-Swin model with cross-frame temporal attention. The DAiSEE KAGGLE dataset’s 6000 video segments were used to extract the original images that capture contextual and face signals. Six augmentation techniques are applied to 12,000 original classroom images of DAiSEE dataset to create an augmented dataset of 84,000 images, which guarantees data diversity and generalization. After that, the data is manually annotated into six academic emotion classes: frustrated, bored, engaged, confused, sleepy, and yawning. The proposed CADE-Swin system outperforms current state-of-the-art algorithms in academic emotion recognition with an impressive 96.13% classification accuracy when applied to annotated images. These findings show that combining dual-stream fusion, context-aware preprocessing, and temporal emotion attention yields a very successful technique to simulate complicated student actions in authentic classroom settings, opening the door for intelligent learning analytics.