DRL-Based Joint Beamforming and UAV Deployment Optimization for STAR-RIS-Assisted Over-the-Air Federated Learning

In this letter, we propose a novel framework of the unmanned aerial vehicle (UAV)-mounted simultaneous transmitting and reflecting reconfigurable intelligent surface (STAR-RIS) communication for over-the-air federated learning (AirFL) systems. The STAR-RIS is deployed onboard the UAV to extend the coverage and improve the link quality between the base station (BS) and distributed clients. To maximize the number of participating clients under an aggregation error constraint, we formulate a joint optimization problem involving the beamforming of the BS and the STAR-RIS, and the UAV deployment. A dual-network hierarchical architecture based on deep reinforcement learning (DRL) is designed to tackle this strongly coupled optimization problem by integrating soft actor-critic (SAC) and double deep Q-network (DDQN). Specifically, the SAC optimizes the beamforming vectors, while the DDQN selects the optimal UAV hovering position from a discrete set of candidates. Simulation results demonstrate that incorporating UAV and STAR-RIS allows more clients to be selected in AirFL and significantly improves federated learning performance.