UAV-assisted beamforming optimization through multi-agent reinforcement learning

This paper explains a communication system that uses unmanned aerial vehicles (UAVs). These UAVs work together to form a virtual antenna array (UVAA). The goal is to improve air-to-ground communication with remote base stations. The paper proposes a way to make the communication more efficient by using collaborative beamforming. The problem is complex because there are two conflicting objectives. First, the system should maximize the transmission rate. Second, it should minimize energy consumption. The positions of UAVs and their signal strengths affect both of these goals. Traditional optimization methods take too long to find a solution. Also, if the situation changes, the old solution may no longer work. To solve this issue, we used Multi-Agent Deep Reinforcement Learning (MADRL). They use a specific approach called HATRPO (Heterogeneous-Agent Trust Region Policy Optimization). Then, they improve HATRPO to create a new version called HATRPO-UCB. Three techniques are added to enhance the learning process. These techniques help the UAVs to learn better strategies. Simulations show that the proposed method outperforms existing techniques. The new approach learns faster and finds better solutions. UAVs can dynamically adjust their positions and transmission parameters in real-time, making the system more adaptable to changes in the environment. By using the reinforcement learning, the UAVs can continuously improve their strategies with less manual tuning. Comparing with traditional optimization techniques, this method significantly reduces the computational cost .Furthermore, the paper also presents a detailed comparison between the proposed method and existing state-of-the-art techniques. The results of our proposed method show better performance in terms of both transmission rate and energy efficiency. To increase the reliability and performance, deep reinforcement learning in UAV-assisted communication networks is applied. The paper provides valuable insights for improving UAV communication systems. In dynamic environments, the important aspect of real-time decision-making can be made using this method. By integrating more advanced reinforcement learning techniques, one can further improve performance and efficiency in optimizing UAV positioning and beamforming strategies.