Live VR Panoramic Video Streaming Method Combing Viewport Prediction and Super-Resolution

The existing VR panoramic video streaming methods based on viewport prediction have not effectively considered the temporal dimension information of VR video, and ignored the role that the client end should play in improving the video reconstruction quality. To further improve the comprehensive performance of the streaming system and the user quality of experience, a live VR panoramic video streaming method that combines viewport prediction and super-resolution reconstruction is proposed. At the server end, the method captures the long-distance dependency in the temporal dimension of video content features by embedding the proposed temporal non-local attention module TNAM into GhostNet to model the global context of VR panoramic video; At the client end, the proposed lightweight VR panoramic video super-resolution reconstruction model LVRSR is used to enhance the quality and optimize the projection distortion of the secondary content within the predicted viewport from the server end. The experimental results on the VR panoramic video user head mo-tion dataset show that the average viewport prediction accuracy and average bandwidth usage of the method are 95.6% and 52.9%, respectively. Compared with five representative streaming methods, the method can achieve higher viewport prediction accuracy and lower bandwidth usage, while having good video reconstruction quality and low computational resource consumption.