Reactive Whole-body Locomotion-integrated Manipulation Based on Combined Learning and Optimization

Abstract Reactive planning and control capacity for collaborative robots is essential when the tasks change online in an unstructured environment. This is more difficult for collaborative mobile manipulators (CMM) due to high redundancies. To this end, this paper proposed a reactive whole-body locomotion-integrated manipulation approach based on combined learning and optimization. First, human demonstrations are collected, where the wrist and pelvis movements are treated as whole-body trajectories, mapping to the end-effector (EE) and the mobile base (MB) of CMM, respectively. A time-input kernelized movement primitive (T-KMP) learns the whole-body trajectory, and a multi-dimensional kernelized movement primitive (M-KMP) learns the spatial relationship between the MB and EE pose. According to task changes, the T-KMP adapts the learned trajectories online by inserting the new desired point predicted by M-KMP. Then, the updated reference trajectories are sent to a hierarchical quadratic programming (HQP) controller, where the EE and the MB trajectories tracking are set as the first and second priority tasks, generating the feasible and optimal joint level commands. An ablation simulation experiment with CMM of the HQP is conducted to show the necessity of MB trajectory tracking in mimicking human whole-body motion behavior. Finally, the tasks of the reactive pick-and-place and reactive reaching were undertaken, where the target object was randomly moved, even out of the region of demonstrations. The results showed that the proposed approach can successfully transfer and adapt the human whole-body loco-manipulation skills to CMM online with task changes.