Deep Reinforcement Learning for AoI-Aware Trajectory and Phase-Shift Design in IRS-Assisted UAV Data Collection

Timely gathering of sensing data is critical in wireless sensor networks (WSNs). However, in delay-sensitive applications, maintaining the freshness of collected data poses a significant challenge. To tackle this issue, an age of information (AoI)-aware data collection method leveraging unmanned aerial vehicle (UAV) and intelligent reflective surface (IRS) is proposed in this work. Particularly, a UAV is employed to traverse over ground sensor nodes (SNs) and reliably collect their sensing data where the received signal strength is enhanced through IRS. The UAV’s flight trajectory and its association with SNs, as well as the IRS phase control strategy are jointly optimized to minimize the weighted sum of the average AoI of the SNs and energy consumption of the UAV. However, this optimization is complicated by potential inaccuracies in IRS channel state estimation. To tackle this challenge, we propose an enhanced deep reinforcement learning (DRL) framework that incorporates a dual-network agent with two nested neural networks (NNs): UAV-NN, which jointly optimizes the UAV trajectory and SN association, and IRS-NN, which dynamically adjusts IRS phase shifts based on sampled channel states, UAV position, and associated SN. By integrating this architecture into proximal policy optimization (PPO) and deep Q-network (DQN), we develop two novel algorithms: PPO-RAC and DQN-RAC, tailored for IRS-assisted UAV data collection. Extensive simulations validate their effectiveness across diverse scenarios, demonstrating significant AoI reduction compared to baseline methods.