Movable Antenna Design for UAV-Aided Federated Learning via Deep Reinforcement Learning

This paper introduces an over-the-air federated learning (OTA-FL) framework that enhances learning efficiency by utilizing movable antennas (MAs) at the access point (AP), with unmanned aerial vehicles (UAVs) acting as federated learning (FL) clients to support Internet of Things (IoT) devices, particularly in remote or disaster-affected areas. We formulate a nonconvex optimization problem aimed at minimizing the Mean square error (MSE) through the joint optimization of antenna placement and beamforming vectors. To address the challenges posed by a dynamic environment, we recast the problem as a Markov decision process (MDP) and propose using the twin delayed deep deterministic policy gradient (TD3) algorithm. Extensive simulations show that the proposed TD3 approach outperforms systems with stationary antennas, including comparisons with fixed-position antennas (FPA) and alternative deep reinforcement learning (DRL) algorithms like soft actor-critic (SAC) and advantage actor-critic (A2C). The results highlight that combining MA arrays with TD3 improves OTA-FL performance and consistently yields higher average rewards, with MA systems proving more efficient than FPA systems.