Visual servoing control of a robotic colonoscope for gastrointestinal examination

Robotic colonoscopy may reduce patient discomfort and operator dependence, but reliable steering remains challenging because intraluminal images contain specular reflections, ambiguous lumen boundaries, occlusion, and anatomy-dependent appearance changes. This paper presents an interpretable low-level vision-guided steering framework that links lumen perception, an explicit image-plane geometric target, and error-dependent gain-scheduled visual servoing. A recurrent residual U-Net trained with segment anything model (SAM)-generated pseudo masks produces a lumen probability map, while an additional confidence head predicts a frame-level segmentation-confidence score. A probability-weighted centroid and five-frame median filter form the visual target, and a reliability gate prevents low-confidence or temporally inconsistent targets from propagating to the controller. When perception is reliable, an error-dependent gain-scheduled proportional–integral–derivative (PID) controller generates yaw and pitch commands to reduce centroid-to-image-center error; otherwise, forward insertion is suspended and the system transitions through temporarily unreliable and recovery states. The framework was evaluated in a collision-free kinematic simulator and a silicone colon phantom as controlled proof-of-concept environments. The segmentation model achieved an inference speed of 30 frames per second with a validation Dice score of 0.92. The mean tracking errors for manual control and the proposed controller were 0.154 and 0.013 cm in a representative Model 1 trial, and 0.0736 and 0.0154 cm across matched simulation trials, respectively. Across three paired phantom trials, the mean image-space error decreased from 201.17 to 136.84 px. These results support the feasibility of an explicit perception–geometry–control architecture for low-level lumen-centered steering.