A Multi‐instrument Recognition and Autonomous Tracking Control Method for Robot‐Assisted Endoscopic Adjustment

This study constructs a robot‐assisted endoscopic adjustment system, which can achieve real‐time precise positioning of surgical instruments (SI) and visual servoing control (VSC) of EFOV under multiple constraints. On the one hand, based on the problem of SI recognition without markers, this study develops a lightweight SI detection model, which improves the detection accuracy of SI in complex surgical environments by introducing an attention mechanism. On the other hand, a hierarchical multiconstraint controller based on image feature points and constraint items is proposed. While achieving VSC, it also realizes constraints on the manipulator joint limits, the endoscope insertion depth, and the remote center of motion. Ultimately, an upper computer software interface is designed, as well as a numerical simulation and an actual SI tracking experiment are carried out. By conducting experiments in these two categories and comparing them with pseudo‐inverse and adaptive control methods, it can be concluded that the average tracking distance are reduced by 35.86% and 28.47% in simulation as well as 13.90% and 11.04% in actual SI tracking experiments, respectively, and constraint items can be effectively controlled and corrected. The results show that this method can adjust the surgical EFOV stably, safely and quickly.