Optimized Visual Servo Control and Digital Twin-Driven Hardware-in-the-Loop Simulation Methods for Autonomous Surface Manipulator Systems

Autonomous surface manipulator systems (ASMS) are novel, versatile robotics platforms consisting of manipulators and unmanned surface vehicles (USVs) on the water. One of their typical tasks is to grasp floating objects using visual servo control (VSC) methods. However, traditional VSC methods use joint velocities of the manipulator as control objects and they cannot be directly applied to ASMS because of the inability to control the manipulators that use joint angles as motion instructions, and the inability to adjust controller parameters based on ASMS's stability during grasping processes. In addition, it is a challenge to test the grasping performance of ASMS safely and efficiently. To address these issues, this article first develops a 12-degree-of-freedom ASMS model that considers the dynamic coupling between the manipulator and the USV, and proposes a normalized stability index for ASMS. Then, an optimized VSC method is proposed, which employs the joint angles of the manipulator as control targets and can adjust the control gains according to ASMS's stability. After that, a digital twin-driven hardware-in-the-loop simulation platform is developed for experiments. The experimental results validate that the optimized VSC method can improve the grasping success rate and stability of ASMS while maintaining high grasping efficiency.