Self-Evolutionary Reinforcement Learning for Autonomous and Robust Multiimpulse Orbital Transfers

Autonomous, robust, and fuel-efficient trajectory optimization is critical for future spacecraft missions, enabling real-time orbital transfers under dynamic mission requirements. <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$N$</tex-math></inline-formula>-impulse transfers between noncoplanar Earth orbits present significant computational challenges for traditional iterative methods, which rely on terminal conditions and lack adaptability to real-time state changes. To address this, we propose a self-evolutionary reinforcement learning (SERL) algorithm that integrates the evolutionary principles of competition and mutual learning from particle swarm optimization into the reinforcement learning framework. This integration enhances the algorithm’s generalization and exploration efficiency. SERL divides the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$N$</tex-math></inline-formula>-impulse transfer process into <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$N-1$</tex-math></inline-formula> intermediate transition orbits, allowing the spacecraft to select intermediate orbits based on its observed states. Compared to traditional methods, SERL offers two key advantages. First, it generates transfer trajectories in real time once the target orbits are defined, enabling onboard deployment and improving spacecraft autonomy. Second, it bases decisions on immediate state observations rather than predefined terminal conditions, allowing it to handle complex tasks, such as dynamically changing target orbits during mission execution. Numerical simulations validate SERL’s ability to improve computational efficiency and adaptability for challenging orbital transfer scenarios.