Hybridizing Long Short-Term Memory Network and Inverse Kinematics for Human Manipulation Prediction in Smart Manufacturing

Human-Robot Collaboration (HRC) is essential for enhancing productivity and flexibility in smart manufacturing, which poses requirements on accurately predicting the future movements of human operators, especially the trajectories of their upper limbs. However, existing model-based studies on human manipulation prediction lacks consideration of stochasticity and variability while the emerging deep learning-based methods are demanding on data size, which yet makes real-time deployment challenging. Therefore, combining the advantages of both model-based and deep learning-based methods, a method for predicting human arm motion, specifically, the position of a worker's wrist in less than 0.5 second, is proposed by hybridizing a Long Short-Term Memory (LSTM) network with an Inverse Kinematics (IK) model. Using historical coordinate sequences of the wrist joint in three-dimensional space in the past multiple frames as input, a neural network is trained to output the predicted coordinates of the wrist joint for the next frame. Then IK (Inverse Kinematics) is used to calculate the arm's motion trajectory based on the predicted wrist coordinates. As the predicted wrist coordinates are sequentially used as the input for the next prediction cycle, the prediction is realized over a sliding time window. Evaluation was conducted using both proprietary and open datasets, results demonstrated that our LSTM-IK method achieved high prediction accuracy, with an average distance error of approximately 5 cm, and can adapt to various task scenarios and individual differences. Additionally, comparison with ground truth illustrated the model's ability to handle complex motion patterns, even with partial occlusions or rapid movements.