Smooth Soft Actor-Critic for Free-Floating Cable-Driven Robot Control

The Free-floating Cable-Driven Robot (FCDR) is an advanced multi-joint rigid robot driven by cable tension, offering high flexibility, low cost, lightweight structure and the ability to carry large loads, making it ideal for space tasks. However, controlling FCDR is challenging due to complex cable dynamics and material properties. Traditional control methods like optimal and fuzzy control require manual adjustments, while adaptive controllers, though robust, are complex and unstable. Recently, deep reinforcement learning (DRL) has shown promise in FCDR control due to its excellent generalization and adaptability. Techniques such as Deep Q-Network (DQN), Deep Deterministic Policy Gradient (DDPG) and Soft Actor-Critic (SAC) have been applied to improve FCDR performance, with SAC standing out for its accuracy and error handling in continuous control tasks. Our approach smooth Soft Actor-Critic (SSAC) combines SAC, target policy smoothing and cost function estimation to enhance trajectory planning stability. Simultaneously, our comparative experiments between Hindsight Experience Replay (HER) and SAC demonstrate that shaped dense reward functions are more effective than sparse reward functions. This work highlights the advancements in DRL for FCDR control, addressing both static and dynamic operational challenges.