Human–Robot Intrinsic Skill Transfer and Programming by Demonstration System

Robots can often learn skills from human demonstrations. Robot operations via visual perception are commonly influenced by the external environment and are relatively demanding in terms of external conditions, manual segmentation of tasks is time-consuming and labor-intensive, and robots do not perform complex tasks with sufficient accuracy and naturalness in their movements. In this work, we propose a programming by demonstration framework to facilitate autonomous task segmentation and flexible execution. We acquire surface electromyography (sEMG) signals from the forearm and train the gesture datasets by transfer learning from the sign language dataset to achieve action classification. Meanwhile, the inertial information of the forearm is collected and combined with the SEMG signals to autonomously segment operational skills into discrete task units, and after comparing with ground truth, it is demonstrated that multi-modal information can lead to higher segmentation accuracy. To make the robot movements more natural, we add arm stiffness information to this system and estimate the arm stiffness of different individuals by creating a muscle force map of the demonstrator. Finally, human manipulation skills are mapped onto the UR5e robot to validate the results of human-robot skill transfer.