Classroom Quality Evaluation System Based on Multimodal Emotion Recognition

Smart classes have become an important trend in education today. The new generation of information technology is driving changes in the way education is delivered. Observing students' emotional state in class will helps teachers accurately knowing how students feel about receiving knowledge, in order to adjust the teaching method. However, the lack of emotional interaction between teachers and students in most realistic classrooms will makes teacher difficult to accurately evaluate students' needs and responses, which affects the quality of teaching. A classroom sentiment evaluation system is proposed based on multi-modal analysis, which adopts vision modal and audio modal to analysis students' emotional state while in class and outputs evaluation scores of classroom quality. The system includes object detection, satisfaction scores analysis, speech recognition, text analysis, and decision feedback module. Additionally, the system will constraint correlation and provides adjustment decision to further improve quality of classroom teaching. This system is verified by real classroom data and obtains a series of experimental data support, achieving satisfactory results of classroom quality evaluation.