A Virtual Reality Framework for Safe Global Bimanual Telepresence

Bimanual telepresence robot control remains the dominant paradigm for completing complex physical work from a safe, remote distance. Additionally, telemanipulation is ubiquitous within training frontier robotic foundation models for fully autonomous, general-purpose manipulation. Despite the contemporary prevalence of dual-handed telemanipulators, few systems ever operate over global-scale distances, with fewer doing so safely. In this paper, a Virtual Reality (VR) framework for safe telepresence control of general bimanual robots is developed. Methods for motion retargeting, low-latency televisualization, and telemanipulation over a simulated distance of 30,000 km are applied to the bimanual robot DOLA. Real-world experiments determine system performance metrics like arm/finger motion tracking accuracy, televisualization latency/acuity, and telemanipulation grasping time/failure statistics. The developed vision system spherically renders a remote scene with 20/100 visual acuity within 80 ms over a Wireless Local Area Network (WLAN) or 300 ms over a Global-Scale Wide Area Network (GS-WAN). Operator arm motions are imitated by a Digital Twin with less than 3% normalized position mean absolute error within 320 ms. Finger motion retargeting with four-fingered robotic hands attains the maximum acheivable Kapandji Score of 9. Lastly, telemanipulation experiments showed 58.4% worsened task completion speed when operating over GS-WAN instead of WLAN. Overall, the developed framework is generalizable to different robots, enabling telemanipulators to be controlled from anywhere at any time.