Automated emotion recognition of students in virtual reality classrooms

In contemporary educational settings, understanding and assessing student engagement through non-verbal cues, especially facial expressions, is pivotal. Such cues have long informed educators about students' cognitive and emotional states, assisting them in tailoring their teaching methods. However, the rise of online learning platforms and advanced technologies such as virtual reality (VR) challenge the conventional modes of gauging student engagement, especially when certain facial features become obscured or are entirely absent. This research explores the potential of Convolutional Neural Networks (CNNs), specifically a custom-trained model adapted from the ResNet50 architecture, in recognizing and distinguishing subtle facial expressions in real-time, such as neutrality, boredom, happiness, and confusion. The novelty of our approach is twofold: First, we optimize the power of CNNs to analyze facial expressions in digital learning platforms. Second, we innovate for the context of VR by focusing on the lower half of the face to tackle occlusion challenges posed by wearing VR headsets. Through comprehensive experimentation, we compare our model's performance with the default ResNet50 model and evaluate it against full-face and VR-occluded face datasets. Ultimately, our endeavor aims to provide educators with a sophisticated tool for real-time evaluation of student engagement in technologically advanced learning environments, subsequently enriching the teaching and learning experience.