NeRFlow: Towards Adaptive Streaming for NeRF Videos

Neural Radiance Field (NeRF) has emerged as a powerful technique for 3D scene representation due to its high rendering quality. Among its applications, mobile NeRF video-on-demand (VoD) is especially promising, benefiting from both the scalability of the mobile devices and the immersive experience offered by NeRF. However, streaming NeRF videos over real-world networks presents significant challenges, particularly due to limited bandwidth and temporal dynamics. To address these challenges, we propose NeRFlow, a novel framework that enables adaptive streaming for NeRF videos through both bitrate and viewpoint adaptation. NeRFlow solves three fundamental problems: first, it employs a rendering-adaptive pruning technique to determine voxel importance, selectively reducing data size without sacrificing rendering quality. Second, it introduces a viewpoint-aware adaptation module that efficiently compensates for uncovered regions in real time by combining pre-encoded master and sub-frames. Third, it incorporates a QoE-aware bitrate ladder generation framework, leveraging a genetic algorithm to optimize the number and configuration of bitrates while accounting for bandwidth dynamics and ABR algorithms. Through extensive experiments, NeRFlow is demonstrated to effectively improve user Quality of Experience (QoE) by 31.3% to 41.2%, making it an efficient solution for NeRF video streaming.