Reinforcement Learning Based Minimum Hop Routing Strategy for LEO Satellite Networks

As satellite networks have grown, it has become more challenging to employ conventional routing algorithms on low-orbit satellite networks because of their huge size and rapid topology changes. Furthermore, the communication efficiency of satellite networks is greatly impacted by the number of routing hops in the end-to-end transmission channel; an excessive number of hops will result in a decline in the network throughput rate and an excessive use of link resources. Consequently, this study suggests a minimum-hop routing strategy for LEO satellite networks based on reinforcement learning. It makes use of the dual-network structure of deep Q-networks (DQN) and a learning strategy, each satellite connects to its four closest neighboring satellites. Then, approaches like epsilon decay, delayed updating of the target network, and empirical playback are employed to help the intelligences in better learning the best policy. Using the Iridium constellation system as a reference, we construct a 6x6 low-orbit constellation model for simulation in order to assess the effectiveness of this approach. The results show that the strategy has good convergence and can effectively reduce the number of satellite routing hops compared to the ant colony routing optimization algorithm. This suggests that the minimum hop count routing strategy based on reinforcement learning has significant advantages in routing optimization for low-orbit satellite networks. The study's findings offer an effective, novel approach to LEO satellite network routing, thereby enhancing the quality of low-orbit satellite networks' communication services and advancing satellite network technology.