PointPresence: An Online Habitat for Multi-User Mixed Reality Telepresence

Mixed reality (MR) telepresence provides a shared common 3D space for networked users to enjoy real-time immersive experiences with natural interactivity and movement. Due to its high hardware and compute demands, a large-scale cloud-edge MR solution is necessary for achieving critical mass adoption, where users engage in the MR environment using low-cost cameras at the edge and complex 3D world processing is offloaded to the cloud servers. However, cloud-edge MR solutions have distinct challenges such as concurrent multiple camera support, unpredictable compute demands, and CPU/GPU contention. In this paper, we present PointPresence, an edge-compute framework for large-scale MR applications. PointPresence is deployed at an edge node, such as an O-RAN RIC or a local enterprise server, and provides low-latency MR experiences to a large number of users by incorporating intelligent camera selection for compute reduction, reactive pipeline design for adaptation to compute demand changes, and context-aware GPU sharing. Our comprehensive evaluation on an MR testbed shows that PointPresence reduces end-to-end latency by up to 3.5× and improves end user perceived visual quality by 30%.