DDS-SLAM: Dense Semantic Neural SLAM for Deformable Endoscopic Scenes

Estimating camera motion and continuously reconstructing dense scenes in deformable environments presents a complex and open challenge. Many existing approaches tend to rely on assumptions about the scene’s topology or the nature of deformable motion. However, these assumptions do not hold true in medical endoscopy applications. To address these challenges, we introduce DDS-SLAM, a novel dense deformable semantic neural SLAM that achieves accurate camera tracking, continuous dense scene reconstruction, and high-quality image rendering in deformable scenes. First, we propose a novel hybrid neural scene representation method capable of capturing both natural and artificial deformations. Additionally, by leveraging the 2D semantic information of the scene, we introduce a semantic loss function based on semantic distance fields. This approach guides network optimization at a higher level, thereby enhancing system performance. Furthermore, we validate our method through a series of experiments conducted on several representative medical datasets, demonstrating its superiority over other state-of-the-art approaches. The code is available at: https://github.com/IRMVLab/DDS-SLAM.