Multipath Cooperative Routing in Ultradense LEO Satellite Networks: A Deep-Reinforcement-Learning-Based Approach

The ultradense low-Earth orbit (UD-LEO) satellite network has attracted significant attention recently due to its great potential in providing global Internet coverage and services. For the sake of improving performance and reliability, multiple network paths can be utilized for coordinated transmission. However, state-of-the-art multipath routing algorithms face the challenge when dealing with highly dynamic network characteristics (i.e., high-speed node movement, frequent topology changes) in such emerging networks. In this article, we propose a deep-reinforcement-learning-based multipath cooperative routing (DRL-MPCR) scheme for UD-LEO satellite networks, with the aim of enhancing routing discovery capability and improving multipath transmission performance. Two main building blocks of multipath transport protocol are considered: 1) routing discovery and 2) multipath scheduling. On the one hand, in order to cope with the highly dynamic satellite network, a DRL-based multipath routing discovery algorithm is proposed, where satellite agents independently make routing decisions according to the perceived local network state, so that multiple available paths can be obtained. On the other hand, to promptly make traffic scheduling according to the varying path conditions, a water filling algorithm-based multipath scheduling policy is designed, which aims to optimize the maximum path cost when multiple paths are utilized for cooperative transmission. Extensive simulation results demonstrate that the proposed DRL-MPCR scheme achieves more efficient routing discovery and better multipath transmission performance than existing ones.