Point cloud matching-based augmented reality surgical navigation system for robot-assisted laparoscopic partial nephrectomy

Abstract: Robot-assisted partial nephrectomy (RAPN) is the standard treatment for localized renal tumors. However, identifying buried tumors and determining resection margins remain challenging. This paper describes a novel augmented reality (AR) surgical navigation system that we are currently developing for RAPNs. The system generates an intra-abdominal point cloud from a stereo endoscopic camera video within approximately 1 second. Point cloud matching is performed between the intraabdominal point cloud and the organ model of the patient created from preoperative computed tomography/magnetic resonance imaging images. We employ a two-stage matching approach: global matching using fast point feature histograms for initial alignment, followed by iterative closest pointrefinement for precise registration. The system successfully achieved registration with an root mean squared error of 1.546 mm for the clinical data testing. By overlaying these point clouds with the matching results, the system enables accurate and intuitive visualization of tumor locations and the positions of blood vessels and other critical structures, even when they are located inside or behind the organs. This AR navigation system has the potential to improve surgical precision, reduce positive margin rates, and preserve healthy renal tissue during RAPN. Keywords: Surgical Support, Robot-Assisted Partial Nephrectomy, Augmented Reality, Point Cloud.