Generalization of Humanoid Arm Manipulation Based on Keypoints Synergetic Guidance

In humanoid robot manipulation imitation learning, arm and tool synergies are required to accomplish tasks. However, the existence of arm and tool shape variations within the demonstrators and between the demonstrator-robot impacts the generalization performance. This paper models the arm and tool as an extended kinematic chain, taking the elbow, wrist, tool end, and tool orientation as keypoints. We propose a humanoid arm manipulation generalization method based on keypoints synergetic guidance, including Postural and Movement Synergy. Using principal component dimensionality reduction and feature space optimization, Postural Synergy obtains the task's start and end positions corresponding to the new target under the new kinematic chain shape. Movement Synergy generates keypoints' trajectories adapted to the new kinematic chain through a dynamical system with coupling terms, and then obtains robot joint trajectories by optimization-based inverse kinematics. Experiments are conducted under shape variations of both the arms and tools. The results indicate that our method significantly reduces keypoints' errors in Postural Synergy compared to Tool-Centric, Motion Retargeting and Cartesian space optimization methods. In Movement Synergy, our method generates dynamically similar motions using coupling terms. Real robot generalization results show that our method is able to improve the task success rate remarkably while mimicking human motion at the same time.