Enhancing UAS Integration in Controlled Traffic Regions Through Reinforcement Learning

Controlled Traffic Regions (CTRs) around major airports pose an important challenge to Unmanned Aerial System (UAS) traffic management. Current regulations highly restrict UAS missions in these areas by confining them to segregated areas. This paper makes a proposal to allow more ambitious UAS missions inside CTRs, such as paths across the CTR or between heliports inside the CTR, based on self-separation. This proposal faces two important problems: on the one hand, the adaptive response to the dynamic airspace reconfiguration of a CTR without necessarily terminating the flight, and on the other, a self-managed conflict resolution that allows maintaining traffic separations without the intervention of air traffic controllers. This paper proposes a solution named Reinforcement Learning Multi-Agent Separation Management (RL-MASM). It employs a multi-agent reinforcement learning system with a fully decentralized decision-making scheme, although it uses a common information source of the environment. The proposed system is evaluated against classical control algorithms for obstacle avoidance to determine the potential benefits of AI-based methods. Results show that AI-based methods can benefit from knowing the intent of a UAS. This leads to increased performance in intrusions into no-fly zones or collisions, and also solves some challenging scenarios for classical control algorithms. From the aeronautical point of view, the proposed solution also introduces important advantages in terms of efficiency, scalability, and decentralization.