Emotion Detection in Digital Classrooms Using Resnet and Hog Feature Extraction Techniques

Maintaining student engagement presents distinct challenges in digital learning environments. This study explores engagement detection to provide actionable feedback for educators and facilitators. The aim is to enhance student participation and interaction in virtual classrooms, offering hope for a more engaging and effective learning experience. A computer vision and deep-learning pipeline was created and trained on the DAiSEE dataset to classify students' levels of confusion and frustration rather than general engagement. Video clips were preprocessed into individual frames, and facial recognition tools were used to detect, isolate, and align the faces. Two feature extraction techniques were independently evaluated: ResNet50V2 and histogram of oriented gradients (HOG). These techniques were chosen for their ability to extract high-level features from images, which is crucial in understanding the complex patterns that compose human emotions. ResNet50V2, a convolutional neural network (CNN), was used with weights pretrained on ImageNet. HOG, a traditional computer vision technique, is well suited for detecting contours and expressions linked to emotional states. Overall, ResNet50V2 achieved 69% accuracy, with an average per-class precision of 26%, per-class recall of 29%, and per-class F1 Score of 25%. HOG achieved a better overall accuracy of 73% but had a lower average per-class precision of 25%, per-class recall of 0%, and per-class F1 Score of 0%. Notably, the models showed stronger performance in frustration detection, achieving between 74-78% accuracy. While these results indicate the system's potential, they highlight key limitations, particularly in handling class imbalance. Future work will include enhancing the precision and recall rates by optimizing the system and adding a weight penalty to reduce the impact of imbalanced data.