A Deep Learning-Based Approach to Arm-Hand Robot Learning from Demonstration

Learning from Demonstration (LfD) is an effective method for equipping robots with manipulation skills, but it faces challenges due to fundamental differences between human and robotic systems. Most existing studies on LfD for arm-hand robots focus on learning arm trajectories, often neglecting the complete manipulation process. To address this limitation, we propose a novel learning framework for arm-hand robots. A depth camera captures datasets of movement trajectories and pose information from human demonstrations. These datasets are processed using Dynamical Movement Primitives (DMP) techniques, enhanced with obstacle avoidance and trajectory smoothing to improve the robot's ability to imitate and generalize trajectory information. An improved motion retargeting algorithm translates hand pose features into joint angle parameters, enabling the generation of diverse object grasping primitives for the robotic hand. A target detection algorithm triggers these primitives, integrating them with arm trajectories to create a comprehensive object manipulation strategy. The proposed framework was validated on the UR5 robotic arm and Inspire dexterous hand, successfully performing object grasping tasks on a desktop and demonstrating its effectiveness in complex manipulation scenarios.