A DRL-Based Spectrum Sharing Scheme for Multi-MNO in 5G and Beyond

In spectrum pooling, which is a well-known technique of spectrum sharing, the initial licensed spectrum of each Mobile Network Operator (MNO) is partitioned into reserved and shared spectrum. The reserved spectrum is for the personal use of an MNO, and the shared spectrum of all MNOs constitutes a spectrum pool that can be flexibly utilized by MNOs that require extra spectrum. Nevertheless, the spectrum pool management problem substantially impacts the spectrum efficiency among these MNOs. In this paper, we formulate this problem as a non-linear programming problem that strives to maximize the average binary scale satisfaction (BSS) of MNOs. To achieve this objective, we introduce an event-driven deep reinforcement learning-based spectrum management scheme, termed EDRL-SMS. This approach adopts a spectrum pool manager (SPM) to efficiently supervise the spectrum pool to reach long-term optimization of network performance. The SPM smartly allocates spectrum resources by fully utilizing a DRL approach, Deep Deterministic Policy Gradient, for each stochastic arrival spectrum request event. The simulation results show that the average BSS of MNOs of the proposed EDRL-SMS significantly outperform our previous work, Bankruptcy Game-based Resource Allocation (BGRA), greedy, random, and without sharing schemes.