Ray-based Multiscale Spherical Grid for Egocentric Viewing

Virtual reality (VR) and augmented reality (AR) demand real-time and high-quality egocentric viewing. Neural rendering can generate extremely high-quality novel views but requires expensive training time and renders too slow. To address these challenges, this paper proposes a method named multi-scale spherical tensor decomposition (MSTD). Its ability to represent egocentric neural 3D scenes also enables potential further applications in relevant domains. Our method can reconstruct scenes from multi-view images or omnidirectional images, outperforming baseline methods in training and rendering time as well as rendering quality.