Enhancing Structure-from-Motion: Re-aligning images with 3D Gaussian splatting

Abstract This paper presents a novel algorithm that enhances Structure-from-Motion (SfM) pipelines by integrating efficient novel view synthesis (NVS) using 3D Gaussian Splatting (3DGS). Traditional SfM methods often fail in challenging scenarios with repetitive structures, outliers or misaligned images, leading to ghost and doppelganger artifacts. To address this, we propose an NVS-guided evaluation and correction framework that leverages 3DGS-rendered views to detect and rectify misaligned images, improving reconstruction accuracy. Our method not only enhances SfM outputs but also benefits downstream tasks such as dense reconstruction [multi-view stereo (MVS)], NVS, and visual localization. Experiments on public datasets and our real-world indoor dataset show that state-of-the-art baselines fail in several ambiguous scenes where our approach succeeds. For example, on the Cambridge Landmarks dataset (Shop Facade), our method reduces localization error from 5.7 to 5.4 cm and improves SSIM from 0.735 to 0.787. These results confirm the robustness and broad applicability of our framework, demonstrating that while 3DGS is an effective tool, the core contribution lies in the proposed SfM re-alignment strategy.