Face Recognition and Emotion Detection in Video Streams using Swin Transformer and EfficientNetV2

In the modern era, face recognition which identifying individuals by analyzing facial features by using algorithms that are trained on huge face datasets, and emotion detection in video streams. The advanced computer vision and Machine Learning (ML) models are used to recognize individuals and infer their emotional states based on facial expressions. However, traditional methods which are struggle with real-time processing in video streams that may lead to poor performance. Therefore, this research proposed a hybrid approach using Swin Transformer and EfficientNetV2 (ST-EFV2) respectively. Initially, the data is collected from both Facial Emotion Recognition 2013 (FER 2013) and Extended Cohn-Kanade Dataset (CK+) that contains vast video streams, facial images and emotional labels. Then, Faster Regional Convolutional Neural Network (R-CNN) is utilized to face detection. Further, ST is employed to extract features as it focuses on key facial expressions and EFV2 assists to classify the emotions. The proposed ST-EFV2 model achieved better results in terms of accuracy with 95.5% compared to Visual Geometry Group-19 (VGG-19) with Residual Network-50 (ResNet-50) model respectively.