FoANet: Focus of Attention Prediction for Foveated Pre-rendering to Enable High-quality Edge VR

Although virtual reality (VR) has garnered significant interest among academia, industry, and consumers, its widespread commercial adoption still faces a critical challenge: providing high-quality visual effects in mobile experiences while ensuring affordability for users. Cloud/edge-based VR applications lead to unacceptable latency, while predictive pre-rendering does not resolve the concern of transmission latency due to the enormous transmission data volume, either. To address this challenge, we introduce a novel foveated pre-rendering solution based on edge VR, which can further decrease latency by reducing the rendering workload and transmission data. We investigate the inherent connection between head and gaze movements in VR and present a multi-task learning model, FoANet, to jointly predict users’ head and gaze movements. Experimental results reveal that our FoANet achieves higher prediction accuracy than separately trained Informer, TimesNet models and SOTA models.