Dynamic Beam Hopping and Resource Management Optimization Based on Deep Reinforcement Learning for Interference Avoidance

The rapid expansion of low-Earth orbit (LEO) constellations has greatly intensified the difficulty of mitigating interference among them. Beam hopping (BH) technology is widely considered as one of the key technologies to achieve on-demand services while avoiding interference among these constellation systems. However, the majority of existing BH schemes neglect to address the interference among separate constellations. This paper proposes a novel BH scheme to tackle interference challenges in multi-LEO constellations coexistence scenarios. In contrast to existing BH schemes that predominantly target intra-system interference mitigation, our proposed BH scheme has the capability to dynamically adjust the beam bandwidth to avoid both intra-system interference and inter-system interference. We formulate an optimization problem that maximizes throughput, guarantees delay fairness, minimizes transmission power for LEO constellations, and avoids inter-system interference. To tackle this formidable non-convex and non-linear problem, the optimization problem is interpreted as a sequential decision-making problem, modeled by a Markov Decision Process (MDP). By utilizing the Advantage Actor-Critic (A2C) algorithm, the BH pattern is dynamically designed to avoid interference between beams from multiple LEO constellations in a time-varying network environment. Simulation results demonstrate that the proposed scheme outperforms other schemes in terms of long-term throughput and delay fairness.