Energy Harvesting and Performance Enhancement of UAV-RIS-assisted Marine IoT: A Deep Reinforcement Learning Algorithm

This study addresses the issues of limited communication coverage and high energy consumption of unmanned aerial vehicles (UAVs)-reconfigurable intelligent surface (RIS)assisted maritime Internet of Things (IoT) systems by proposing an optimization framework based on deep reinforcement learning (DRL). Specifically, a dual UAV-RIS cooperative maritime communication model is first constructed, where two UAVs equipped with RIS modules serve multiple maritime user terminals (UTs). First, we formulate a jointly optimization problem to maximize system throughput and energy harvesting efficiency via optimizing UAV trajectories, RIS phase shifts, base station (BS) power allocation, and energy harvesting duration. Given the nonconvexity and complex coupling constraints of the problem, we remodel the joint optimization task as a Markov decision process (MDP), and further proposes a numerical estimation of multiple Q-values soft actor-critic (NEMQ-SAC) algorithm, which significantly improves learning performance in high-dimensional action spaces. Simulation results demonstrate that the proposed method significantly outperforms traditional approaches in terms of energy harvesting efficiency and system throughput, validating its potential in complex and dynamic maritime environments.