PMSNeRF: Patch-Based Neural Radiance Fields for Generalizable Multi-View Reconstruction

Neural Radiance Fields (NeRF) has achieved notable advancements in neural reconstruction and rendering, but it faces challenges with long rendering times for each scene and struggles to generalize new scenes. We introduce a new multi-view reconstruction technique called PMSNeRF. This method significantly improves the ability to generalize and the quality of novel view synthesis. By incorporating the shallow layers of the Visual Geometry Group (VGG) network to extract patch features from target views and enhancing local features during training, our model is better equipped to capture differences in images, leading to improved reconstruction outcomes. Drawing inspiration from a Two Time-Scale Update Rule (TTUR), we perform patch extraction every 64 steps, which boosts training efficiency and optimizes overall model performance while making effective use of computational resources. Our results show that PMSNeRF surpasses various generalizable NeRF models across multiple synthetic and real-world datasets, demonstrating clear advantages in managing rendering artifacts and complex geometric structures. The methodology we propose holds significant potential for applications in various domains, including robotics, virtual reality, and augmented reality.