Energy Efficient PSO Integrated TD3 Reinforcement Learning with Nonlinear Energy Harvesting in UAV Assisted D2D Networks

Device-to-Device (D2D) communication has become essential to 5 G and beyond 5 G networks due to its enhanced spectral and energy efficiency capability. In order to support unmanned aerial vehicles (UAVs) in D2D communication, energy efficiency (EE) has become an essential metric. This paper proposes a novel approach for optimizing energy-harvesting in D2D networks overlaid with UAV using simultaneous wireless information and power transfer (SWIPT). The proposed method leverages the strengths of two powerful algorithms: particle swarm optimization (PSO) and twin-delayed deep deterministic policy gradient (TD3). At first, PSO is employed to optimize the D2D nodes within the network, maximizing energy harvesting opportunities while satisfying Quality-of-Service (QoS) requirements. Subsequently, the optimized D2D node configurations are fed into a TD3 agent, which learns the optimal power allocation method for UAV to D2D and D2D to D2D transmissions. The advantages of using TD3 in proposed model are reduction of overestimation, enhanced stability, fast convergence and better exploration. In D2D network overestimation reduction helps to get better reward in achieving energy and throughput, the second advantage is stability which refers to the network’s ability to maintain consistent and reliable data transmission over time, the third advantage is fast convergence which makes the D2D network to reach the objectives i.e. energy efficiency and throughput in less amount of time and the last advantage of TD3 is better exploration, during search operation of network when the D2D node is selected for transmission every time the entire network is searched. The qualities of PSO and TD3 make the system achieve better energy efficiency and throughput. Simulation results validate that the proposed model particle swarm optimized TD3 (PSOT) achieved better results when compared with the prevailing algorithms.