Virtual Muscle-Model-Modified Reinforcement Learning for Tilt Motion Control of a Novel Magnetic Actuated Flexible-Joint Robotic Surgical Camera System

The magnetic actuated flexible-joint robotic surgery (MAFRS) camera system enhances laparoscopic surgeries by extending operational periods, achieved through the elimination of onboard motors. However, current methods face challenges in providing precise tilt motion control due to the variability in abdominal environments and the complexity of magnetic field interactions. To overcome these challenges, we propose a virtual muscle-model-modified reinforcement learning (RL) approach. This approach employs the deep deterministic policy gradient algorithm, optimized for continuous action spaces, thereby improving system robustness and response to nonlinear dynamics. The virtual muscle concept, drawing inspiration from human musculature, is integrated to mitigate camera chattering within the RL framework. Our system demonstrates exceptional control precision across various abdominal wall thicknesses, maintaining accuracy within <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathbf {0.2^{\circ }}$</tex-math></inline-formula>–a value approaching the resolution limit of our sensors. This level of precision signifies a significant advancement in laparoscopic robotic technology.