Efficient Packet Routing for Large-Scale LEO Satellite Networks: A Pareto-Optimal MARL Approach With Queueing Theory

Low Earth orbit (LEO) satellite networks enhance terrestrial connectivity by providing global coverage and low-latency communication. However, their highly dynamic topology, time-varying propagation delays, and constrained bandwidth severely limit the efficiency of conventional centralized routing, underscoring the necessity for adaptive and distributed strategies that can operate effectively under partial observability. Multi-agent reinforcement learning (MARL) offers a promising foundation for such strategies by enabling decentralized, context-aware decision-making based on local information. Nevertheless, existing MARL-based routing approaches often struggle to maintain accurate congestion awareness, reconcile conflicting objectives, and ensure stable convergence in large-scale LEO constellations. To address these challenges, we present POMAP, a packet routing framework that integrates Pareto optimization with multi-agent proximal policy optimization (MAPPO) to achieve efficient and stable trade-offs across multiple key performance metrics. Specifically, we propose a dynamic multi-attribute graph model for LEO satellite networks that simultaneously captures communication delay and energy consumption. Within this framework, each satellite node is represented as a G/G/1/K queue equipped with active queue management and scheduled using weighted priority queueing, thereby enabling precise characterization and control of packet queueing behavior. We formulate the packet routing problem as a partially observable Markov decision process that jointly minimizes delay, energy consumption, and packet loss rate, and apply MAPPO to optimize the resulting policy. Extensive simulations on realistic satellite network topologies demonstrate that the proposed method achieves better convergence stability, improved Pareto front coverage, and enhanced overall network performance compared with state-of-the-art baselines.