GestRight: Understanding the Feasibility of Gesture-driven Tele-Operation in Human-Robot Teams

In this paper, we propose GestRight, a real-time system for gesture-based tele-operation of a mobile robot. For field use (e.g., smart factory settings, search and rescue missions, etc.), relying on tablet-based controls or joysticks are limiting which has led to the recent interest in hands-free operation of these assistive robots. In this work, we design three gesture-based schemes, namely, fist, touch, and wheel, represent three levels of precision–intuitiveness tradeoffs for low-level navigational control of mobile robots. GestRight includes a head-mounted device that captures hand joint data for accurate gesture recognition which is then translated to motion commands at an edge server. Through a user study involving seventeen participants, we present quantitative insights in comparison to traditional modes of control. Specifically, we evaluate GestRight in terms of the ease of navigational control, task time, and amount of errors/corrective actions required, run extensive statistical analyses, and provide a series of design recommendations for gesture-driven teleoperation systems. Our results show that gesture based schemes perform as well as traditional modes of control in contrast to participants’ self-reports on how successful they felt in controlling the robots.