Deep deterministic policy gradient based routing protocol for UWSNs

Underwater Sensor Networks (UWSNs) monitor the oceans, detect seismic events, and explore the environment. However, these networks also suffer from several limitations, including dynamic topology, propagation delay, limited bandwidth, limited energy resources, and the problem of void nodes. The void node issue occurs when nodes lack next-hop neighbors to forward data packets, resulting in data loss and increased energy consumption. To address these shortcomings, this paper introduces D2PG-RP, a Deep Deterministic Policy Gradient-Based Routing Protocol for UWSNs, to optimize network performance and energy consumption. D2PG-RP uses an actor-critic network to optimize routing decisions to minimize energy consumption during data packet transmission. The proposed approach incorporates a reward function based on critical parameters, including depth information, propagation delay, residual energy, Node Mobility Factor (NMF), Node Reputation Score (NRS), and Acoustic Interference Level (AIL) to mitigate the impact of the void node problem, and select stable, reliable nodes for data forwarding without excessive noise. We perform the simulation in MATLAB 2024b. The results of our simulation study for underwater sensor networks demonstrate that D2PG-RP outperforms legacy Q-learning models in terms of energy consumption, network lifetime, end-to-end delay, and packet delivery ratio.