Intraoperative fusion of models and data for robust distance sensing

PURPOSE: Instrument-integrated optical sensors are gaining popularity in microsurgery due to their ability to provide accurate measurements of instrument-to-tissue distances, enabling precise instrument control. However, obstructions in the optical path can result in measurement errors. In this work, we propose a method to improve robustness of distance information from sensorized microsurgical instruments. METHODS: Our pipeline integrates a rapid search algorithm to identify relevant neighboring data points, as well as geometric and non-geometric techniques to accurately model the local tissue structure. Additionally, we implement a fusion of measurement and model to identify and overcome disturbances, e.g., obstructions from surgical instruments or semantic segmentation errors. RESULTS: Our simulation examines the effect of different modeling parameters and techniques on distance prediction, yielding a mean absolute error of less than 0.02 mm when using the local spline fit. Experiments in ex vivo human eyes show that our pipeline achieves up to 89 % error reduction when compared to sensor only. CONCLUSION: Our method improves the reliability of instrument-integrated optical sensors. This work could enable distance-based instrument control in challenging conditions, thereby enhancing surgical precision in delicate ophthalmic procedures. Our approach can be generalized to any surgery with sensorized instruments and beyond.