WMSDs Motion Recognition and Evaluation Method Based on Multimodal Fusion and Deep Learning

In response to the severe impact of work-related musculoskeletal disorders (WMSDs), this study proposes an advanced motion rehabilitation solution. By integrating inertial measurement unit (IMU) sensors and MediaPipe skeletal point technology, we aim to deliver personalized and precise motion assessment and rehabilitation guidance for WMSDs patients. Capturing and analyzing the shoulder and hip movements of 130 volunteers, we constructed a multimodal dataset with 30[Formula: see text]600 sets of data. A hybrid model combining one-dimensional convolutional neural networks (1D-CNN) and gated recurrent units (GRUs) was employed, achieving 97% accuracy in action recognition through dynamic dropout and L2 regularization. Additionally, an automatic scoring mechanism based on deep learning and Chinese limb motor function standards was introduced to enhance motion rehabilitation. The innovative application of multimodal fusion, deep learning scoring, and an efficient hybrid model not only improves scoring efficiency and consistency but also contributes new technologies and methods to the field of motion rehabilitation for WMSDs. Future work will focus on improving data collection and processing techniques, exploring new model structures and algorithms, and developing customized rehabilitation plans to meet individual needs.