Multiple Sensor-based Adaptive Foveated Display Control for Enhanced Computational Efficiency of Virtual Reality Devices

The demand for immersive virtual reality (VR) experiences in high resolution has escalated, necessitating the development of advanced hardware (HW) capable of rendering images with high quality.To optimize the computational efficiency of delivering high-quality images, it becomes imperative to align with the constraints of the human visual system.The foveated rendering approach strategically allocates rendering resources by prioritizing the highest quality at the user's gaze point while reducing image quality in the periphery.In this paper, we propose an adaptive foveated display controller algorithm that leverages a grid-based strategy and seamlessly integrates with varying levels of detail to dynamically construct the foveated rendering technique.Our technique dissects images into grid cells while tracking the user's gaze through an eye-tracking sensor.The resolution is highest at the grid cell where the user is looking, gradually decreasing in nearby grid cells.This efficient approach reduces the need for heavy computing and effectively minimizes rendering latency, which is commonly found in non-foveated rendering methods.We conducted experiments using our proposed approach on several hardware.This setup allowed us to gather data from a range of interconnected sensors and run the adaptive foveated display controller algorithm.The results of our experimentation demonstrated a significant reduction in both latency and resource consumption, clearly outperforming the non-foveated rendering approach.