Endo-E2E-GS: End-to-end 3D reconstruction of endoscopic scenes using Gaussian Splatting

Three-dimensional (3D) reconstruction is essential for enhancing spatial perception and geometric understanding in minimally invasive surgery. However, current methods like Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) often rely on offline preprocessing—such as COLMAP-based point clouds or multi-frame fusion—limiting their adaptability and clinical deployment. We propose Endo-E2E-GS, a fully end-to-end framework that reconstructs structured 3D Gaussian fields directly from a single stereo endoscopic image pair. The system integrates (1) a DilatedResNet-based stereo depth estimator for robust geometry inference in low-texture scenes, (2) a Gaussian attribute predictor that infers per-pixel rotation, scale, and opacity, and (3) a differentiable splatting renderer for 2D view supervision. Evaluated on the ENDONERF and SCARED datasets, Endo-E2E-GS achieves highly competitive performance, reaching PSNR values of 38.874/33.052 and SSIM scores of 0.978/0.863, respectively, surpassing recent state-of-the-art approaches. It requires no explicit scene initialization and demonstrates consistent performance across two representative endoscopic datasets. Code is available at: https://github.com/Intelligent-Imaging-Center/Endo-E2E-GS . • We propose Endo-E2E-GS, a fully end-to-end framework that reconstructs 3D Gaussian fields from stereo endoscopic images without requiring scene-specific initialization. • The framework integrates a DilatedResNet-based stereo depth estimator, a Gaussian attribute predictor, and a differentiable splatting renderer for photometric supervision. • Experiments on ENDONERF and SCARED datasets demonstrate superior reconstruction quality and highlight the framework’s robustness in texture-scarce surgical environments. • The method enhances geometric perception in endoscopic surgery and holds promise for integration into robotic-assisted clinical workflows.