Adaptive 360-Degree Video Streaming with Super-Resolution and Interpolation

360° video streaming requires considerable bandwidth, and many techniques have been proposed to address this problem. One such technique is super-resolution, where the video is compressed at the server, and the client runs a deep learning model to enhance the video quality. However, most of today’s off-the-shelf mobile devices cannot support super-resolution for all tiles in real time. As a result, some tiles cannot be reconstructed to high resolution, significantly reducing users’ Quality of Experience (QoE). To address this problem, we utilize linear interpolation, which requires much less computational overhead. Through experiments, we observe that interpolation can achieve comparable quality, and even outperform super-resolution for some tiles with low spatial complexity. Building on this, we develop a 360° video streaming system that adaptively selects the most suitable downloading strategy, whether interpolation, super-resolution, or ABR at the appropriate bitrate, for each tile to maximize user QoE while considering network bandwidth limitations and the computational constraints of mobile devices. We formalize the 360° video streaming problem as an optimization problem and propose an efficient algorithm to solve it. Extensive evaluations using real user viewing data and 5G network traces demonstrate that our solution significantly outperforms existing techniques in terms of QoE under various scenarios.