Improving SLAM‐Based Navigation in Flexible Ureteroscopy by Kidney Stone and Surgical Tool Segmentation

Flexible ureteroscopy is a widely used surgical procedure for diagnosing and treating various urinary tract conditions, particularly kidney stones. Ensuring the complete extraction of all stones is crucial to prevent recurrence and the need for auxiliary interventions. Visual SLAM-based navigation systems have been proposed to assist surgeons by simultaneously estimating the 3D structure of the kidney and tracking the ureteroscope's tip position. However, most existing solutions assume a completely static environment, which does not account for the intraoperative situation. In this study, we extend the work of Oliva Maza et al. by incorporating real-time visual segmentation of kidney stones and surgical tools using either YOLOv7-E6E and segment anything or YOLO11m-seg. Our method discards pixels corresponding to instruments due to their inherent dynamic nature, while kidney stone pixels are incorporated into the SLAM framework but classified as potentially dynamic map points, allowing for their disappearance. This refinement enhances the robustness and the accuracy of ureteroscope position estimation for surgical navigation. To evaluate our approach, we recorded multiple datasets for both segmentation and ureteroscope pose estimation. Experimental results show an average improvement in ureteroscope pose estimation of 35.4% when using YOLOv7-E6E with SAM, and 52.49% when using YOLO11m-seg.