Enhancing Human-Robot Teleoperation through Depth Sensing and eXtended Reality

Teleoperation enables the control of robots in remote, complex, or hazardous environments such as manufacturing, healthcare, and disaster response. However, traditional teleoperation systems lack immersive depth perception, which limits spatial awareness and precision. This paper presents a novel teleoperation framework integrating an Intel RealSense D345i depth camera with the Reachy humanoid robot, enhanced by augmented reality (AR) overlays in a virtual reality (VR) interface. By leveraging depth sensing, point cloud processing, and AR visualization, the system improves task accuracy, spatial awareness, and user experience. Our system utilizes advanced frameworks for seamless AR and VR integration. Preliminary findings demonstrate enhanced performance in tasks requiring precise object manipulation, with future user studies planned for quantitative evaluation.