NeRVis: Neural Radiance Field Model-Uncertainty Visualization

Neural Radiance Field (NeRF) is a promising deep learning approach for three-dimensional (3D) scene reconstruction and view synthesis, with various applications in fields like robotics and medical imaging. However, similar to other deep learning models, understanding NeRF model inaccuracies and their causes is challenging. The 3D nature of NeRFs further adds challenges such as identifying complex geometrical features and analyzing 2D views that suffer from object occlusions. Existing methods for uncertainty quantification (UQ) in NeRFs address the lack of NeRF model understanding by expressing uncertainty in model predictions, exposing limitations in model design or training data. However, these UQ techniques typically rely on quantitative evaluation that does not facilitate human interpretation. We introduce NeRVis, a visual analytics system that supports model users to explore and analyze uncertainty in NeRF scenes. NeRVis combines spatial uncertainty analysis with per-view uncertainty summaries, fostering analysis of the uncertainty in Lambertian NeRF scenes. As a proof-of-concept, we illustrate our approach using two UQ methods. We demonstrate the effectiveness of NeRVis with two different use scenarios, tackling key challenges in the NeRF UQ literature. • We propose NeRVis, a novel visual analytics approach for exploring uncertainty in NeRF scenes. • NeRVis combines spatial and per-view uncertainty visualizations for analysis. • We demonstrate NeRVis’ effectiveness with two scenarios based on UQ-NeRF literature challenges.