Depth map compression, streaming, and reconstruction for immersive and accessible 3D telepresence

The recent surge in remote work and digital communication has highlighted limitations in traditional video conferencing, creating demand for more immersive alternatives. Despite advances in 3D hardware, widespread adoption of 3D telepresence applications remains limited due to challenges in encoding, compressing, and transmitting 3D data over standard connections. This dissertation addresses these challenges through novel techniques for 3D range geometry compression, streaming, and reconstruction. We introduce HoloKinect, a user-friendly 3D video conferencing platform using off-the-shelf hardware that leverages multiwavelength depth (MWD) encoding and standard video codecs for efficient transmission. The limitations of MWD motivated the development of N-DEPTH, a neural depth-to-RGB encoding scheme optimized through a differentiable JPEG approximation layer. N-DEPTH demonstrates superior compression resilience and lower reconstruction error by learning an encoding strategy robust to compression artifacts. Addressing the computational cost of neural methods, we introduce GraDE (Gray Depth Encoding), a computationally efficient algorithm inspired by N-DEPTH, MWD, and Gray codes. GraDE achieves competitive or superior performance to previous approaches, particularly for video streaming, offering a lightweight alternative for resource-constrained devices. Finally, our Collaborative Spatial Streaming platform demonstrates a multi-device application using GraDE, enabling dynamic, spatially coherent 3D capture and streaming from commodity mobile devices in immersive AR/VR environments. Overall, this research advances the state of the art in 3D telepresence by providing efficient, accessible, and robust solutions for depth compression and reconstruction.