Efficient Packet Routing in Ultra-Dense LEO Satellite Networks via Cooperative-MARL with Queuing Theory Model

Advances in communication technology, coupled with the growing global demand for network connectivity, have established low Earth orbit (LEO) satellite networks as a critical complement to terrestrial networks. However, the inherent high latency, dynamic topology, and bandwidth limitations of LEO satellite networks, along with the inefficiencies of centralized routing strategies, pose significant challenges to the effectiveness of conventional terrestrial routing protocols. Traditional routing methods, which rely on static, rule-based approaches, lack the flexibility required to adapt to dynamic network conditions, underscoring the necessity for more adaptive packet routing strategies within satellite network protocols. To address these challenges, we propose a packet routing optimization algorithm that integrates cooperative multi-agent proximal policy optimization (MAPPO) with the M/M/1/K queuing theory model. Specifically, we develop a multi-attribute graph model for dynamic satellite networks that incorporates both communication delay and energy consumption metrics. For each satellite node, we establish an M/M/1/K queuing model that accounts for the queuing waiting time at each node. We formulate the packet routing optimization problem, aiming to minimize delay and energy consumption, as a partially observable Markov decision process and apply the multi-agent proximal policy optimization algorithm to refine the routing policy. Extensive simulations on real satellite network topologies demonstrate that the proposed algorithm significantly outperforms existing methods, achieving higher cumulative rewards while reducing both latency and energy consumption.