Multiagent DRL for Two-Timescale Bandwidth Allocation in Multibeam Satellite Networks

Multibeam technology, as one of the crucial technologies for implementing high throughput satellite (HTS) systems, can flexibly allocate the beam resources to enhance communication quality and transmission rates. Due to the variability in service demand, traditional resource allocation methods struggle to accommodate the dynamic changes in user service requirements. Thus, heuristic algorithms, such as the genetic algorithm (GA) and deep reinforcement learning (DRL) are commonly used in current research for communication resource allocation. However, existing resource allocation strategies mainly focus on radio resource allocation in a single timescale, neglecting the differentiated time granularity resource scheduling requirements arising from the distinct characteristics of variability in beam-level and user-level resource demands. In this article, we investigate bandwidth allocation strategies in multibeam satellite (MBS) networks. Considering the potential drawbacks of single-agent algorithms in terms of the high complexity for bandwidth allocation in dynamic user demands, a two-timescale hierarchical bandwidth allocation algorithm based on multiagent DRL is proposed to minimize the total system delay and guarantee the delay fairness for ground users. Simulation results demonstrate that the proposed algorithm requires shorter training time to converge and exhibits superior performance in communication delay and system fairness compared with the single-agent DRL algorithm.