DDPGRU: Enhancing DDPG with a GRU-Based Actor Network for Capturing Temporal Dependencies in State Dynamics

Deep reinforcement learning (DRL) empowers agents to learn complex behaviors through dynamic interaction with the environment, and has been widely applied in fields such as robotics, autonomous driving, and gaming. As one of the popular and widely applied DRL algorithms, Deep Deterministic Policy Gradient (DDPG) combines the powerful function approximation capabilities of neural networks with deterministic policy gradients, successfully addressing reinforcement learning challenges in continuous action spaces. However, DDPG often fails to capture the temporal dependencies of dynamic states in real-world environments, leading to suboptimal policies and poor generalization performance. In this paper, the Gated Recurrent Unit (GRU) is incorporated as the backbone network of the Actor to learn the temporal dependencies of state dynamics. Based on this enhancement, an improved algorithm called DDPGRU is proposed to address the limitation of insufficient temporal modeling capabilities. The key innovation of DDPGRU is that DDPG provides a robust framework for continuous control, while GRU enables the agent to simulate and leverage temporal patterns within the environment. Experimental results demonstrate that the proposed DDPGRU algorithm significantly outperforms the original DDPG baseline algorithm.