When Rewards Deceive: Counteracting Reward Poisoning on Online Deep Reinforcement Learning

Deep Reinforcement Learning (DRL) agents are particularly vulnerable to poisoning attacks, where adversaries subtly manipulate reward signals to alter agent behavior toward a specific undesirable policy. In response, this paper introduces a novel defense mechanism that leverages a multi-environment training strategy, significantly enhancing the resilience of DRL agents. This strategy is underpinned by a non-cooperative Bayesian game model, which captures the dynamic interplay between the DRL agent and its attacker. To further strengthen agent defenses, we incorporate a variance-based detection method that identifies reward manipulations by establishing a critical decision threshold. Our experimental evaluation involves rigorous testing through simulation-based experiments, which validate not only the theoretical robustness of our approach, but also its practical effectiveness across diverse DRL scenarios. This research provides a comprehensive blueprint for building resilient DRL agents capable of maintaining optimal performance, even when faced with sophisticated adversarial challenges.