Multimodal Deep Learning-Driven Biomechanical Modelling of Sports Movements and Real-Time Sports Training Intervention System

In this paper, we propose a novel multimodal deep learning-based biomechanical modelling of sports movements and real-time sports training intervention system, aiming to break through the bottleneck of high cost and scene limitation of traditional motion capture technology. The system integrates an inertial measurement unit (IMU), a multi-view RGB-D camera array and a ground reaction force measurement platform, and innovatively designs a spatio-temporal attention-enhanced graphical convolutional network (STAGCN) and a cross-modal comparative learning framework (CMCL), to achieve efficient extraction and precise alignment of multi-dimensional motion features (temporal synchronisation error <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\leq 1 ~\text{ms}$</tex>, spatial error <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\leq \pm 2 ~\text{mm}$</tex>). Combined with the meta-learning-driven personalised parameter self-adaptation mechanism, the system can complete the fast adaptation of new athletes with only 5 times of movement data, and the knee moment prediction error is reduced by 35.4 % (from 8.2 Nm to 5.3 Nm) compared with that of the generic model. Experiments show that the system achieves 94.2 % action recognition accuracy and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{9 2. 4 \%}$</tex> injury prediction sensitivity in basketball jumping action analysis, and the delay of the whole process is controlled within 17 ms (7 ms for model inference), and the power consumption is as low as 3.5 W. The system provides a high-precision, low-latency and intelligent solution for the optimisation of competitive sports training, the prevention and control of risks in mass fitness, and the assessment of sports rehabilitation.