Two-stage relationship modeling algorithm for student classroom behavior recognition

The objective of student classroom behavior recognition is to employ artificial intelligence technology for locating and identifying student behaviors within the classroom. Publicly available datasets in this domain are relatively scarce due to concerns about student privacy. Furthermore, current research lacks a clear standard for defining student classroom behavior and encounters limitations in modeling its influencing factors. To tackle these issues, this paper constructs a dataset named SCBD (Student Classroom Behavior Dataset) and proposes a two-stage relationship modeling network (TSRM-Net). TSRM-Net employs the YOLOv7 algorithm to locate student positions. In the first stage of relationship modeling, it utilizes an adaptive 3D convolutional neural network to automatically learn the direct relationship between students and the classroom environment. In the second stage of relationship modeling, it employs a visual attention mechanism to extract interaction information among students. Finally, behavior classification is conducted with a multilayer perceptron network that utilizes the fused features obtained from the two-stage relationship modeling. The proposed approach outperformed traditional methods, achieving the most favorable experimental results on the SCBD dataset.