SCB-DETR: Multiscale Deformable Transformers for Occlusion-Resilient Student Learning Behavior Detection in Smart Classroom

The integration of artificial intelligence (AI) into the modern educational system is rapidly evolving, particularly in monitoring student behavior in classrooms—a task traditionally dependent on manual observation. This conventional method is notably inefficient, prompting a shift toward more advanced solutions such as computer vision. However, existing target detection models face significant challenges such as occlusion, blurring, and scale disparity, which are exacerbated by the dynamic and complex nature of classroom settings. Furthermore, these models must adeptly handle multiple target detection. To overcome these obstacles, we introduce the student classroom behavior detection with multiscale deformable transformers (SCB-DETR), an innovative approach that utilizes large convolutional kernels for upstream feature extraction, and multiscale feature fusion. This technique significantly improves the detection capabilities for multiscale and occluded targets, offering a robust solution for analyzing student behavior. SCB-DETR establishes an end-to-end system that simplifies the detection process and consistently outperforms other deep learning methods. Employing our custom student classroom behavior (SCBehavior) dataset, SCB-DETR achieves a mean Average Precision (mAP) of 0.626, which is a 1.5% improvement over the baseline model’s mAP and a 6% increase in AP50. These results demonstrate SCB-DETR’s superior performance in handling the uneven distribution of student behaviors and ensuring precise detection in dynamic classroom environments. The source code of this study is publicly available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/CCNUZFW/SCB-DETR</uri>.