Sign potential-driven multiplicative optimization for robust deep reinforcement learning

Deep Reinforcement Learning (DRL) has attracted the interest of researchers due to its ability to provide valuable solutions to a variety of problems in different fields, such as robotics, autonomous driving, financial trading, and more. However, DRL models are often sensitive and exhibit unstable behavior during both training and evaluation, increasing the interest in implementing robust methods to enhance the DRL training process. Taking into consideration that optimization methods based on multiplicative updates have shown to leverage advantages over additive ones in terms of convergence and robustness, we propose a novel momentum-based optimization approach that overcomes significant limitations of existing methods, such as the inability of multiplicative updates to flip the sign of parameters. More specifically, the proposed approach employs a sign-change mechanism, inspired by spiking neural networks, allowing parameters to change signs benefiting in that way the training process in terms of learning acceleration and robustness. The proposed optimizer is oriented to train DRL agents and is experimentally evaluated in various tasks, demonstrating its effectiveness in the training of DRL agents.