DDPG-Ent: A Novel Integration of Maximum Entropy in Deep Reinforcement Learning

Progress in deep reinforcement learning (DRL) has demonstrated the effectiveness of its algorithms in continuous action spaces. In particular, the Deep Deterministic Policy Gradient (DDPG) algorithm adopts an actor-critic network architecture, optimizing the parameters of both networks to adjust the actor network's output, enabling it to accomplish complex tasks. However, this approach of optimizing via policy gradient can easily fall into local optima, as DDPG is a deterministic policy that always selects a specific action, leading to insufficient exploration, especially in complex or high-dimensional continuous action spaces. In this paper, we propose a method called DDPG-Ent, which introduces an entropy term to encourage the policy to maintain a certain degree of randomness in the action space. This helps explore more state-action combinations in the environment, preventing early convergence to local optima.