Deep Reinforcement Learning for Adaptive Traffic Engineering in Satellite Constellation Networks

With increasing demand for broadband services provided by satellite constellation networks, routing and traffic management have become increasingly relevant topics. To enable global coverage and end-to-end connectivity, these systems rely on inter-satellite links to span space-borne networks. To fulfill the quality of service requirements of significant network loads, the traffic load needs to be balanced optimally. As rule-based techniques to solve the underlying multi-commodity flow problem are too complex to comply with on-board processing power limitations, specifically tailored light-weight solutions are required. To this end, we propose an adaptive traffic engineering approach based on deep reinforcement learning. We explore a policy-based approach for a flexible flow allocation between candidate paths. We compare the schemes with state-of-the-art benchmarks based on heuristics, and an optimal benchmark using linear programming. The results highlight that the proposed scheme is able to approximate optimal solutions to the multi-commodity flow problem and can learn suitable policies for diverse sets of paths. Moreover, after training, the approach exhibits low complexity in inference, and is thus well-suited to be included in the controller logic of in-space distributed software defined networks.