A federated reinforcement learning framework for balancing rapidity and availability in deep space networks

Abstract Efficient task scheduling is essential for the success of deep space exploration missions, where communication delays, dynamic link availability, and limited on-board resources pose significant challenges. However, existing deep-space scheduling frameworks cannot effectively coordinate multi-agent decisions across interplanetary regions due to long delays, dynamic link conditions, and highly unbalanced resources, resulting in inefficient and unstable task allocation. To address these issues, we propose Fed-ASTRA, a novel scheduling framework that integrates a hierarchical deep space network architecture with federated multi-agent reinforcement learning. In the proposed architecture, base stations such as Earth and Mars ground centers act as regional control centers, orbital satellites function as edge intelligence nodes, and rovers serve as terminal execution units, forming a three-tier collaborative system. This hierarchical organization enables both local autonomy and cross-domain coordination, effectively balancing real-time responsiveness and long-term availability. On the algorithmic side, we model the scheduling process as a multi-agent Markov decision process and design environment-constrained action pruning (ECAP) to filter out infeasible actions caused by link outages, energy thresholds, and deadline violations. In addition, a prioritized experience replay (PER) mechanism improves sample efficiency by emphasizing high-cost task experiences. These enhancements ensure that agents learn feasible and efficient scheduling strategies under severe environmental constraints. We evaluate Fed-ASTRA using a high-fidelity deep space network simulator driven by real orbital data. Experimental results demonstrate that our framework outperforms traditional optimization approaches, heuristic methods, and baseline MARL algorithms in terms of rapidity, availability, fairness, and real-time capability. Overall, this work highlights the potential of combining hierarchical network architectures and federated reinforcement learning to achieve robust and efficient task scheduling in future deep space missions.