GESTRO: Skeleton-Based Dynamic Gesture Recognition for Robot Teleoperation

Abstract Lately, improving the efficiency of human-robot interaction (HRI) has gained significant importance, particularly in industrial settings, where the robot’s ability to understand and act on human gestures has become essential for a smooth collaboration. This paper contributes to the challenge of real-time gesture recognition for mobile robot teleoperation. It introduces a new tailored dataset GESTRO with 11 dynamic gestures captured in 3,300 samples in total dedicated for intuitively controlling mobile autonomous systems. Based on the proposed dataset, this paper introduces a LSTM (long short-term memory) model with attention mechanism, achieving an accuracy of 0.98. An extensive evaluation examines the data quality and the importance of dynamic sequence for accurate gesture detection. Finally, the proposed model is implemented on a mobile robotics system, demonstrating its usability in realistic human-robot interaction setting. The proposed dataset along with the presented models are made publicly available ( https://github.com/thohemp/gestro ).