A Control Framework for a Robotic Endoscope Holder Based on Lightweight Foundational Segmentation Models

Minimally Invasive Surgeries (MIS) present significant challenges due to the limited field of view (FOV), constrained motion range, and the reliance on manual endoscope operation, which can lead to fatigue and unstable imaging. Robotic-assisted endoscope systems address these issues by providing precise, stabilized positioning with constrained motion control. In this work, we propose a control framework for a robotic endoscope holder, incorporating a training-free method based on lightweight foundational segmentation models for surgical instrument localization, and a Quadratic Programming (QP) controller for motion planning under constraints. The framework ensures adherence to the Remote Center of Motion (RCM) constraint for patient safety, while image-based visual servoing enables real-time autonomous tracking of surgical instruments. The proposed method is validated using a 7-DOF Rokae Pro3 robotic arm with a rigid endoscope, achieving real-time performance with an RCM error consistently within 4.49 mm and smooth trajectory tracking. Experimental results demonstrate the framework’s effectiveness in addressing the challenges of constrained motion planning in MIS, enabling safe, precise, and autonomous endoscope positioning and instrument tracking.