Multifeature fusion viewport prediction method based on transformer

The continuous advancement of virtual reality and Internet technology has gradually integrated 360° video into daily life. With its characteristics of high immersion and interactivity, 360° video offers an immersive viewing experience that has garnered widespread attention. However, the rich content inherent in 360° videos results in large amounts of data, imposing significant demands on the network environment required for video transmission. Given the fixed field of view of headmounted displays (HMDs), theoretically, only a small part of the video content needs to be guaranteed to ensure a seamless viewing experience and significantly reduce bandwidth consumption. Therefore, accurate viewport prediction is crucial. In this paper, we propose a 360° video viewport prediction framework based on multi-feature fusion using Transformers to address the limitations of existing methods, which often focus on a single type of data and exhibit low long-term prediction accuracy. Our approach leverages three different types of data: the user's past head movement trajectory, video saliency, and cross-user interest areas, to achieve more robust viewport prediction. Experimental results demonstrate that, compared to existing methods, our approach more accurately predicts future viewport areas and significantly improves the accuracy of long-term predictions.