Non-Contact Orientation Mapping of Robot Manipulator for Human-Robot Interaction using Depth Sensor

Since the depth sensor provides the joint position data of human arm, it is commonly used for the position mapping of robot manipulator in human-robot interaction. However it is difficult to be used for orientation mapping. The reasons include the failed fingertip detection cannot be used to estimate the hand pose due to self-occlusion between fingers, and the low fingertip positioning accuracy of the depth camera cannot meet the control requirements of the robot. In this paper, a novel human hand pose estimation based on cascading deep neural network is proposed to solve the above problems. In the experiment, the optical tracking sensor data is used as a reference, and a real robot platform is used to verify the effectiveness. The results show that the orientation motion of the operator's hand is carried out within 50°, the proposed method can accurately realize the orientation mapping for robot manipulator, and the estimation accuracy of orientation mapping is less than 5°.