Research on intelligent detection and intervention strategies of silent students in university classroom based on multimodal behavior recognition

This study aims to solve the problems of the lag of traditional detection methods and the lack of personalized guidance of intervention measures by constructing intelligent detection and intervention strategies for silent students in universities based on multimodal behavior recognition. In this study, an improved cross-modal Transformer model (CM-Trans) is adopted, which integrates visual, auditory and textual information, and effectively improves the detection accuracy of silence behavior. The experimental results show that the CM-Trans model performs well in the multimodal fusion ablation experiment. Compared with the single-mode and dual-mode combination, the indexes of [email protected] and F1_silence are significantly improved, reaching 0.83 and 0.86 respectively, and the performance is further improved to 0.85 and 0.87 after the antagonism training. The design of three-level dynamic intervention mechanism gives consideration to technical effectiveness and humanistic care. The combination of real-time micro-intervention, medium-range adjustment and long-period counseling effectively improves classroom participation and knowledge retention rate. After the fourth week, the participation rate of group A remained above 75%, the gain of participation duration and the knowledge retention rate reached +12.3 minutes/class and +22.5% respectively, and the change of anxiety value was minimal. In contrast, the intervention effect of group B and group C, which were only tested by the model, was limited, and the anxiety value of group C increased significantly. Group D without intervention showed a slight downward trend. The ethical risk assessment shows that students' acceptance of the system is generally high, with the overall value identification score of 4.3/5.0, intervention comfort score of 4.1/5.0, privacy concern and anxiety trigger score of 3.2/5.0 and 2.8/5.0 respectively, indicating that students have some concerns about privacy protection, but the intervention measures have not caused anxiety significantly.