Spatiotemporal Energy Harvesting Optimization in UAV-RIS Networks via PER-SD3

This paper investigates the energy efficiency optimization problem of a Unmanned Aerial Vehicle-Reconfigurable Intelligent Surface (UAV-RIS)-assisted wireless communication system, where the nodes rely on energy harvesting for operation. The core challenge is to maximize long-term energy harvesting efficiency while maintaining communication quality, complicated by spatiotemporal channel and energy dynamics. We propose a robust deep reinforcement learning (DRL) framework named Prioritized Experience Replay Soft Double Q-learning (PER-SD3). We introduce the Hybrid Time Switching (HTS) model and the Reconfigurable Allocation Hybrid Time-Space (RAHTS) model. These models are designed to accurately capture the spatiotemporal dynamics of energy harvesting and signal reflection in the UAV-RIS network. In addition, we explore multiple UAV trajectory designs, including hovering, Fermat point navigation, and K-means-based mobility strategies. These trajectory designs are comprehensively evaluated under diverse scenarios to understand their impact on energy harvesting efficiency and communication performance. The simulation results show that our proposed PER-SD3 outperforms existing DRL baselines, such as Soft Double Q-learning (SD3), Twin Delayed Deep Deterministic Policy Gradient (TD3), and Deep Deterministic Policy Gradient (DDPG), in higher energy harvesting efficiency and faster convergence. The RAHTS model with adaptive trajectory control enhances both energy efficiency and node fairness, offering a practical solution for sustainable UAV-RIS networks.