Prediction of joint moment in lower limbs based on deep learning and multimodal data

This study aimed to predict the moments at the hip, knee, and ankle joints in multiple planes during various movements. A deep learning model was developed using a bidirectional long short-term memory network (BiLSTM) combined with an agent attention mechanism (AA). Multimodal data were collected from 20 young subjects, including anthropometric data, joint angles, electromyographic signals, and ground reaction forces. The corresponding joint moments were calculated using Anybody motion simulation software. These data were used as input to the BiLSTM-AA model for joint moment prediction. Different input combinations and dimensionality reduction methods were compared. The best results were obtained by integrating anthropometric data, joint angles, and ground reaction forces. In cross-subject tests, the model showed high accuracy, with a mean absolute error of 0.0395 Nm/kg, root mean square error of 0.0579 Nm/kg, and a coefficient of determination of 0.9117. The model also performed well after reducing input dimensions. In summary, the BiLSTM-AA model predicts lower limb joint moments accurately across activities. This may simplify real-world data collection and help provide solid evidence for rehabilitation planning and assessment. • A novel BiLSTM-AA model is proposed for multi-joint and multi-plane moment prediction. • The model demonstrates high accuracy and strong generalization in cross-subject tests. • Joint angles combined with ground reaction forces form the most effective data combination. • High predictive performance is maintained after Random Forest and PCA dimensionality reduction.