Detection of fragmentation while dusting during retrograde intrarenal laser lithotripsy: a novel computer vision and perception pipeline

Dusting during ureteroscopy with laser lithotripsy is popular because it minimizes the need for basketing of fragments. However, one of the challenges with dusting is adjusting power settings to be efficient and to limit the inadvertent generation of fragments. Detection of fragment production in real time using computer vision could help facilitate energy adjustments to optimize dusting. With IRB approval, we recorded eight consecutive ureteroscopy with laser lithotripsy procedures performed for intrarenal stones with a single-use flexible ureteroscope. We used a Dornier Thulio laser to dust the stones, with power settings at the discretion of the surgeon. We included all patients with stones between 7 mm and 15 mm. Time and duration of clips showing fragmentation were labeled. We then developed a fragmentation during dusting detection pipeline based on contemporary AI technologies, including semantic segmentation and video point tracking, and qualitatively demonstrated the detection performance on paradigmatic video segments. Successful extraction was performed in all 8 surgical videos, but only 5 had sufficient fragmentation events and adequate visualization for inclusion. We evaluated our pipeline’s performance on frame-level fragmentation detection, achieving an F1 score of 0.568, driven by a recall (sensitivity) of 0.516 and a precision (positive predictive value) of 0.632. Segment-level fragmentation detection performance was substantially lower, with an F1 score of 0.124 due to low precision (0.072) despite a recall of 0.438. We also qualitatively analyzed our pipeline on representative segments and provide insight into the task-specific challenges. Recent advancements in video point tracking enable more accurate capture of object motion patterns, which could be used to track the fracture of fragments from the kidney stone. With refinement, the proposed pipeline for the detection of fragmentation may be utilized to provide feedback to the surgeon and/or laser to adjust power settings in real time.