A Multi-Agent Reinforcement Learning Based Cooperative Beam Hopping Scheme for LEO Mega Constellation Networks

The LEO (Low Earth Orbit) mega constellation network with beam-hopping scheme has become a promising approach to provide global communications. Most of the research on the beam-hopping scheme design for LEO constellation network has focused on beam-hopping scheduling and power allocation for a single LEO satellite. Few of them considered the cooperation between adjacent satellites covering the overlapping areas. As a result, many satellites illuminate beams to the same areas covered by multiple adjacent satellites wasting the limited onboard resource, and the users located in single coverage areas cannot be served. In this paper, we design a multi-agent reinforcement learning (MARL) based cooperative beam-hopping scheme for the LEO mega constellation network, where the LEO satellites cooperatively schedule the beam-hopping and allocate resource. Specifically, we formulate the beam-hopping problem as a multi-agent model, and propose a Centralized Training and Decentralized Execution (CTDE) algorithm. Various simulations are conducted to evaluate our proposed scheme and the results show that the cooperative beam-hopping outperforms the existing schemes in terms of resource utilization and system throughput.