Routing in Low Earth Orbit Satellite Networks: Constrained DQN-Based Approach

The complexity of routing in low earth orbit satellite networks is increasing due to higher traffic loads, complex network architectures and their integration with the terrestrial networks. The time varying satellite network topology, inter-satellite link failures and the necessity to satisfy diverse quality of service requirements demands the need to investigate routing algorithms that can fulfill multiple objectives. Value based reinforcement learning methods such as Q-learning facilitate a mechanism for interaction, learning and decision making within centralized and distributed environments. In this work, we design a constrained deep Q-network based routing algorithm that calculates optimal routing paths by ensuring safe exploration. We evaluate the performance of the proposed routing algorithm for different mesh cluster sizes and identify conditions where the constraints cannot be satisfied, necessitating a switch to rule-based source routing algorithms. Furthermore, we discuss resiliency aspects such as learning in the event of dynamic link failures, computational complexity, and scalability for large satellite constellation networks.