Autonomous Space Systems via Reinforcement Learning: Review, Challenges, and Future Directions

Reinforcement learning (RL) is increasingly being investigated for spacecraft guidance, navigation, control, and mission autonomy, particularly where uncertainty, nonlinear dynamics, limited onboard computation, or changing operating conditions challenge conventional approaches. This review examines RL and closely related learning-based methods across major space-control applications, including attitude and orbit control, low-thrust and cislunar guidance, planetary landing, small-body operations, rendezvous and docking, space robotics, surface mobility, and mission-level autonomy. Reported results are assessed with respect to classical baselines, quantitative performance, robustness, computational cost, and validation fidelity from simulation to hardware and flight demonstrations. Across the literature, learned controllers often show advantages in adaptation, robustness to modeled uncertainty, and rapid onboard evaluation, while numerical optimization and classical control generally retain advantages in nominal optimality, interpretability, and established guarantees. Hybrid architectures, in which learning modifies or augments a conventional guidance or control law, provide some of the most consistent results. Important limitations remain in out-of-distribution generalization, benchmarking, statistical reporting, navigation realism, verification, implementation assurance, and independent high-fidelity validation. The review concludes with proposed benchmark problems, reporting practices, and a staged validation pathway for assessing learned controllers in increasingly realistic space-system environments.