Energy Efficiency Maximization in RIS-Assisted UAV Networks: An MSAC-Based Approach

This paper focuses on an unmanned aerial vehicle (UAV)-assisted reconfigurable intelligent surface (RIS) system in a multiple-input single-output (MU-MISO) wireless communication network. The aim is to maximize the energy efficiency of the UAV-RIS by jointly optimizing the UAV trajectory, the base station transmit beamforming matrix, and the UAV-RIS reflection coefficient matrix under imperfect channel state information (CSI). Considering the complexity of the proposed optimization problem, the Multi-agent Soft Actor-Critic (MSAC) reinforcement learning algorithm is adopted for solution. When constructing the system model, the channel models under perfect and imperfect CSI are analyzed in detail, and the UAV energy consumption model is also described. The optimization problem is transformed into a multi-agent reinforcement learning problem, and the state space, action space, and reward settings are clearly defined. Simulation results show that the energy efficiency is better under perfect CSI conditions, and the system tends to be stable as the training period increases. Under imperfect CSI conditions, the reward values with different error levels gradually approach each other, demonstrating the robustness of the MSAC algorithm in handling such problems and verifying the effectiveness of the proposed method.