A Binocular Vision-based Method for Real-time 3D Localization of Surgical Instrument End-Effectors

Real-time 3D localization of surgical instrument end-effectors is of paramount importance in Minimally Invasive Surgery (MIS), as it enhances surgical precision and enables quantitative skill assessment and training. However, determining the instrument’s spatial position from intraoperative imaging remains a significant challenge. To address this, a method is proposed for instrument tip tracking that leverages a binocular stereo vision system. First, image processing algorithms are employed to detect the 2D pixel coordinates of the instrument tip from stereo images captured during surgery. Subsequently, through stereo rectification and correspondence matching, the 3D coordinates of the tip are reconstructed. Experimental validation, using an optical positioning system as the ground truth, demonstrated that the trajectory tracking achieved a Root Mean Square Error (RMSE) of approximately 2.34 mm on average across multiple trials. These results afrm that the proposed method provides an accurate and viable solution for real-time end-effector localization, holding significant potential for advancing surgical navigation and performance evaluation.