Deep Viewpoint Prediction in 360º Video with Data Distribution Adaptation

Tile-based 360º video transmission approaches in-volve accurate prediction of the user's viewpoint trajectory. Current models still exhibit significant prediction errors for videos with high viewpoint dispersions, increasing the likelihood of low-quality of user experience. This issue may be attributed to the considerable variations in users' viewpoint data distribution when watching such videos. To address this problem, this paper proposes a deep viewpoint prediction model in 360º Video (DVP360). The model extracts spatio-temporal features of both image and viewpoint information through convolutional neural networks (CNN). It then employs Long Short-Term Memory (LSTM) networks to contextually combine these spatio-temporal features. Finally, we adjust the model parameters through knowledge transfer, which enhances the model's ability to adapt to variable data distributions, thereby improving prediction performance. Numerical results indicate that DVP360 outperforms baseline methods on the MMSYS18, CVPR18, and PAMI18 datasets, where especially, the prediction error is reduced by 18.7% on the MMSYS18 datasets.