Self-Attention-Based Deep Reinforcement Learning for Joint Beamforming and Phase Shift Design in Aerial Irs Networks

This paper investigates the joint design of the transmit beamforming matrix and the intelligent reflecting surface (IRS) phase shift matrix in a multi-user, multiple-input singleoutput (MU-MISO) system integrated with an aerial IRS. The proposed approach leverages the high mobility and flexibility of unmanned aerial vehicles (UAVs) to enhance the probability of a line-of-sight (LoS) link, thereby maximizing the sum spectral efficiency of the user equipments. Specifically, to adapt to the time-varying nature of real-world communication environments, we propose a twin delayed deep deterministic policy gradient (TD3) algorithm incorporating self-attention mechanisms for the joint optimization of beamforming and phase shift strategies. Furthermore, the batch normalization technique is employed to improve the algorithm's capability to process extensive state and action spaces, thereby accelerating convergence. Simulation results demonstrate that the proposed algorithm outperforms other mainstream deep reinforcement learning (DRL)-based baseline methods.