Obstacle-Avoidance Distributed Reinforcement Learning Optimal Control for Spacecraft Cluster Flight

The scientific objectives of space missions impose stringent requirements on the cooperative control systems of spacecraft clusters. Achieving optimal distributed consensus control, particularly under external disturbances and collision avoidance, remains a significant challenge. To address this issue, the problem of spacecraft cluster flight cooperative control using Reinforcement Learning (RL) and adaptive control with collision-free guarantees is investigated for rigid spacecraft clusters subjected to system uncertainties and unexpected disturbances. Initially, obstacle-avoidance performance functions are employed to impose desired performance metrics on the collision-free behavior of the spacecraft cluster. Specifically, the smaller the long-term index, the greater the distance from obstacles, thereby ensuring enhanced safety. The challenge of collision avoidance in spacecraft is reframed into a "Min-max collision-free Actor-Critic" RL mechanism. This framework effectively enhances collision avoidance capabilities by minimizing the maximum long-term indexes (Critic) within the spacecraft cluster through an optimized control strategy (Actor). Subsequently, a fully distributed control strategy is derived by integrating convex optimization, adaptive control, actor neural networks (NN), backstepping control, and the maximum consensus algorithm. It is demonstrated that the designed optimal distributed cooperative controller can achieve control targets without collision risk, while ensuring the boundedness of all closed-loop signals, as verified through the Lyapunov stability theorem and convergence analysis of infinite series. Finally, simulation experiments on spacecraft cluster flight confirm the effectiveness of the proposed control protocol.