MetaSplats : Rapid Sparse 2D View to 3D Novel View Synthesis

In this work, we introduce a unique methodology to address the challenges associated with novel view synthesis (NVS) in 3D computer vision, specifically in situations with sparse views and varying poses. Usually the intrinsics and extrinsics of the camera are estimated using Structure-from-Motion (SfM) pipelines in conventional methods, but these techniques can be slow and unreliable, especially when faced with a lack of matching features. Our approach, Metasplats, combines end-to-end dense stereo models like DUSt3R with the benefits of point-based representations combined with optimized 3D Gaussian splatting (3D-GS). Our goal is to effectively manage the intricacies of neural network systems in bounded scenes like 3D objects. The usefulness of MetaSplats is demonstrated by experimental results on datasets such as Mip-NeRF360. Our approach maintains quality while significantly accelerating processing speeds with just four input images, marking a substantial leap in efficiency. These results confirm that MetaSplat is a workable solution in NVS applications for bounded scenes when pose-free and sparse-view conditions are present. Metasplats introduces a novel pipeline enabling users to upload four to six images from either a mobile device or camera, facilitating the rapid generation of good-quality 3D models.