Integrating Confidence Maps and Visual Servoing for Needle Tracking in Robotic US-Guided Percutaneous Nephrolithotomy

Ultrasound-guided percutaneous nephrolithotomy (PCNL) is a minimally invasive procedure to remove large kidney stones through an incision in the patient’s back. PCNL requires a high level of dexterity to steer a surgical tool while visualizing it using ultrasound images. A robotic system that controls the ultrasound probe to automatically image the tool would alleviate the surgeon’s cognitive workload and potentially lead to more accurate kidney access. We propose a novel algorithm that combines visual servoing and confidence maps to track the position of a manually steered needle using a robotically actuated ultrasound probe. The algorithm automatically adjusts the position of the ultrasound probe so that the same longitudinal portion of the needle shaft is visible in the image, while simultaneously ensuring acoustic contact between the ultrasound probe and the tissue over uneven surfaces. Unlike previous methods, where confidence maps were used for probe positioning with static targets, this paper introduces the first unified algorithm that optimizes image quality while tracking a moving tool. It ensures continuous probe-tissue contact on uneven surfaces and does not require prior knowledge of the needle’s trajectory or additional sensors. The algorithm, evaluated in phantom tissue and in a realistic kidney mannequin, shows an average tool tracking accuracy of 1.65 mm and 1.17 mm, respectively, confirming its ability to reliably track a manually inserted tool during PCNL.