BUT telepresence: a fully open-source VR framework for low-latency stereoscopic video streaming with end-to-end latency instrumentation

Abstract Advances in immersive telepresence have enabled compelling virtual reality(VR)-based teleoperation, yet existing systems remain difficult to reproduce,inflexible in their streaming configuration, and largely devoid of end-to-endlatency instrumentation. Current research prototypes typically rely on fixedencoding parameters, do not expose a unified interface for real-time reconfig-uration, and rarely support hardware-accelerated decoding on standalone VRheadsets. As a result, the community lacks an open, validated reference frame-work for low-latency stereoscopic teleoperation. This paper introduces a fullyopen-source VR telepresence system that integrates stereoscopic capture ona two-degrees-of-freedom (2-DOF) camera module, low-latency encoding usingGStreamer, head-coupled camera actuation, and a native OpenXR applicationcapable of hardware-accelerated decoding on modern standalone VR headsets.A key contribution is a built-in end-to-end latency instrumentation mechanismthat timestamps each stage of the pipeline and embeds these measurementsinto the media stream, enabling continuous monitoring by the VR client. Thisinformation enables the user to dynamically adjusts codec, resolution, andframe rate to maintain stable operator perception under heterogeneous networkconditions, including wireless links. The framework is validated in representa-tive robotics scenarios, demonstrating reliable stereoscopic teleoperation withmeasured end-to-end latencies down to approximately 50–80 ms in optimized1configurations and around 80-100 ms under constrained wireless conditions,with the VR headset’s pose prediction helping to mitigate perceived delay forthe operator. All software, including benchmark datasets and Grafana dash-board configuration, is released publicly at https://github.com/Robotics-BUT/BUT Telepresence to support reproducibility and future research.