Compression and Transmission of 8K Stereoscopic VR Using VAE-GAN Latents and Standard Encoders

Despite the heightened popularity of Virtual Re-ality (VR), streaming high-resolution stereoscopic VR remains a challenge. This is primarily due to significant bandwidth demands of high-definition VR content. While advanced deep neural networks (DNN s) have demonstrated the potential to outperform standard codecs, their integration into real-world transmission frameworks is complex and not directly compatible with current encoding standards. To bridge this gap, this paper proposes a novel technique for compressing and transmitting 8K stereoscopic scenes using Variational Autoencoder (VAE) GAN latents represented as 3-channel RGB scenes that can be transmitted via standard encoders. The proposed method reduces bandwidth requirements by 45.1 % across different 8K scenes while maintaining visual quality, highlighting the effectiveness of the approach. This study also investigates the impact of varying patch-sizes of input frames for model training and evaluate its influence on client-side reconstructions. We then explore various transmission configurations of latent frames. Our findings suggest that while residual transmission offers limited benefits for 3-channel latent frame compression, raw transmission consistently yields better results, particularly for texture-heavy scenes. To the best of our knowledge, this is the first such transmission study on 8K stereoscopic scenes for cloud-based VR, providing valuable insights for optimizing high-resolution VR streaming systems. Code: github.com/sampreetucalgary07/8K-VR-compression.