Advanced Facial Expression Classification with CNN-Transformer Integration for Human-Computer Interaction

This paper presents an advanced approach to Facial Expression Classification (FEC) to evaluate user behavior in online shopping environments. In this application, users' videos are captured as they accomplish tasks under varying circumstances, including scenarios with and without moderator aid. We utilized and trained a simplified POSTERv1 model on the AffectNet dataset to analyze the captured videos. The model processes frames and performs first, face detection b y using the MTCNN approach. Then, the detected face is resized and normalized to ensure compatibility with the input requirement of the deep learning architecture. The normalized face image is fed to the facial landmark detector and facial feature extractor networks. The outputs from these two parallel pipelines are provided to the cross-fusion transformer encoder to capture multi-scale features and enhance expression recognition accuracy. Experimental results demonstrate the model's efficacy, achieving notable accuracy across AffectNet, CK+, and FER2013 datasets. Our approach effectively addresses real-world challenges in FEC by creating a custom dataset and comparing emotional responses in moderated versus non-moderated scenarios, highlighting its potential for Human-Computer Interaction applications.