Reinforcement Learning-Based Dynamic Random Access Optimization Scheme for 5G Space TT&C System

With the development of 5G non-terrestrial networks (NTN) and aerospace technologies, tracking, telemetry, and command (TT&C) users will establish connections with TT&C stations through Random access (RA) in future space TT&C systems based on 5G. However, due to the limited RA resources and their uniform allocation among beams, the existing RA scheme may result in severe preamble collisions and access unfairness when dealing with bursty user traffic and unbalanced loads across different beams, leading to numerous access failures and increased user delays. In this paper, we aim to jointly consider access control and RA resource reallocation for each beam by using the Proximal Policy Optimization(PPO) method, which outputs dynamic Access Class Barring (D-ACB) factors and beam resource allocation factors. Unlike existing methods that require predicting the number of backlogged users, we introduce Long Short-Term Memory (LSTM) networks for training, which only utilize the observable historical access results to learn the policy. Simulation results demonstrate that our optimization scheme nearly approaches the performance of ideal upper bound, and significantly outperforms the baselines in terms of the number of successfully accessed users, access delay, and collision probability at least by 11.83%, 21.73%, and 64.82%, respectively.