Designing Gesture-Based Interfaces for Cobots: Insights from a Mixed Reality Elicitation Study

The rapid evolution of human–robot interaction, particularly within human–robot collaboration, presents a transformative opportunity through mixed reality (MR) technologies to redefine industrial automation and safety. This article addresses the increasing demand for intuitive human–robot interfaces in emerging paradigms like Industry 5.0 and the Industrial Metaverse. Our motivation is to bridge the gap in understanding how users naturally interact with collaborative robots when immersed in MR environments. We present a novel elicitation study utilizing a Microsoft Hololens 2 head-mounted display (HMD), where users perceived their gestures directly controlling a cobot through real-time visual feedback within the MR space. This methodology allowed us to derive a set of natural, intuitive gesture-based control mechanisms tailored for MR-enhanced industrial robotics. The novelty lies in providing empirically grounded insights into user-defined gestural language under MR constraints for cobot navigation tasks and exploring the influence of user characteristics on these gestures. Our findings demonstrate MR’s potential to foster a more integrated, productive, and safe workspace by enhancing operational efficiency, reducing operator learning curves, and improving safety protocols in high-risk settings. We also identify key differences in gesture selection based on gender (in the gesture’s “Nature”) and professional background (in the gesture’s “Locale”), providing design implications for user-adaptive interfaces. These findings advance the applicability of MR technology in complex industrial environments, proposing new paradigms for human–robot interaction.