Resource Allocation for Ultra-Dense 5G and Beyond Networks Based on Proximal Policy Optimization

In recent years, resource allocation in Ultra Dense Networks (UDNs), which consist of Small Base Stations (SBSs), plays a crucial role in enhancing the efficiency and capacity of the network. However, significant challenges such as severe interference, inefficient resource utilization and congestion are introduced in the network due to the high density of SBSs. To overcome these challenges, a policy-based reinforcement learning method, Proximal Policy Optimization (PPO) is proposed, which continuously learns and adapts the network conditions by adjusting power levels and resource allocations, ensuring maximum Spectral Efficiency (SE) and Energy Efficiency (EE). Firstly, a 5G UDN downlink framework is developed, which includes a Macro Base Station (MBS). This MBS is integrated with a server, namely Mobile Edge Computing (MEC), which performs resource allocation among distributed SBSs and User Equipment (UEs). Furthermore, PPO is employed to train the MEC, which acts as an agent and interacts with the whole UDN environment, to make better decisions by improving its policy through reinforcement learning updates. The proposed PPO outperformed DQN by achieving 800bps/Hz in SE and 700bps/J in EE respectively.