RoBERTa-based Multimodal Sentiment Analysis of Detecting Teachers’ Emotions

Multimodal sentiment analysis, which combines data from various sources such as text, images, and audio, has become a vital tool in educational settings for monitoring teachers’ emotions in the classroom and ensuring teaching quality. We propose a novel model called RoMSA (RoBERTa-based Multimodal Sentiment Analysis) to enhance the effectiveness of this process. RoMSA utilizes a new multimodal joint embedding mechanism that fully captures cross-modal information, and it incorporates a Random Modal Masked Language Modeling (RMMLM) task to improve the model’s understanding of different modalities. By leveraging the extensive textual knowledge obtained during pretraining, RoMSA significantly improves the accuracy of multimodal sentiment analysis. Tested on the CMU-MOSI and CMU-MOSEI datasets, RoMSA achieved state-of-the-art results, demonstrating its effectiveness in detecting and analyzing emotions. Through the analysis of teachers’ emotions during classes, multimodal sentiment analysis models can output teachers’ emotional states, helping to ensure teaching quality and enabling educators to optimize their teaching methods to maintain a positive classroom atmosphere.