Laparoscope Adjustment with Stereo Vision for Robot-Assisted Surgery Based on Multi-task Control

Assisted laparoscope-holding robots (ALHRs) offer enhanced precision and stability in minimally invasive surgery (MIS). They cooperate with the surgeon by automatically tracking the surgical instruments. However, existing autonomous laparoscope adjustment methods (ALAMs) lack 3D perception capabilities, leading to risks of blurred imagery and collisions. Additionally, there are issues with accumulated remote center of motion (RCM) error and hand-eye incoordination. Therefore, this paper proposes a binocular ALAM based on multi-task control for robot-assisted surgery (RAS). To overcome the limitations of existing researches, the overall framework of the system is designed and its kinematic model is established. Then, an instance segmentation-based surgical instrument segmentation Network (ISSINet) is proposed for accurate and efficient segmentation of multi-type and multi-scale surgical instruments. Furthermore, instrument tip matching rules are proposed to achieve 3D perception of the tips. Additionally, a compensator is designed to eliminate the accumulated RCM error. Finally, a binocular ALAM method based on multi-task control is proposed. Compared with the existing methods, the mean Dice coefficient of ISSINet improved by 2.9 points, representing a 3.04% increase over the baseline. The simulation results show that the binocular ALAM converges effectively. In real machine experiments, the mean RCM error decreased from 0.62 mm to 0.04 mm, and the mean orientation error decreased from 0.17 rad to 2.56×10<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-4</sup> rad. Experimental results demonstrate that the proposed method significantly enhances the safety and stability of ALHRs.