Toward the Age in Forwarding: A Deep Reinforcement Learning Enabled Routing Mechanism for Large-Scale Satellite Networks via Spatial–Temporal Graph Neural Networks

In large-scale satellite networks, end-to-end data delivery confronts long propagation latency and dynamical connectivity, heavily challenging current routing strategies due to the time-changing topology. For the tasks with high timeliness requirements, especially, it will cause obviously low freshness of data since huge computational cost and transmission overhead are required for implementing optimal routing decisions. To address this issue, in this paper, we propose an Age-driven Deep Reinforcement Learning (DRL) enabled Routing Mechanism (ADRLRM) for the large-scale satellite network by considering the impact of time-varying topology on the timeliness of data, in which a Parameter-adjustable Two-step Slotting Algorithm (PTSA)-assisted Spatial-Temporal Graph Neural Network (STGNN) is well-designed to extract the topological features both in temporal and spatial dimensions. In particular, we develop a novel metric of data freshness called Routing-aware Age of Information (RAoI) to evaluate the timeliness of data delivery, which is incorporated into the customized reward function of the proposed mechanism as an optimization objective. The simulation results indicate that the proposed mechanism performs better in reducing the end-to-end delivery latency, number of forwarding hops, and average RAoI compared with the conventional Open Shortest Path First (OSPF) routing algorithm, Deep Q-Networks based Intelligent Routing (DQN-IR), and GNN-based GraphPR routing algorithm, respectively.