Adversarial Reinforcement Learning with MPGD for Worst-Case Perception Error Simulation at Highway On-Ramps

Reinforcement learning (RL) has demonstrated considerable promise in the field of autonomous driving. However, decision-making processes in real-world environments are inevitably influenced by measurement noise and perception inaccuracies, which can result in suboptimal or even unsafe actions. To mitigate these risks, it is imperative for autonomous vehicles to model and account for such perception uncertainties effectively. This study focuses on the ramp merging scenario in autonomous driving, where environmental uncertainties are modeled as adversarial perturbations to the states observed by RL agents. We propose a novel adversarial attack framework that combines RL with momentum-based projected gradient descent (MPGD), aiming to simulate worst-case perception errors by perturbing the sensory inputs of the driving agent. Experimental evaluations across varying traffic densities and multiple RL algorithms for autonomous driving demonstrate that our approach outperforms three baseline adversarial attack strategies in simulating the worst-case perception errors. Additionally, adversarial training of the driving agent with our attack model significantly enhances the robustness of the autonomous vehicle, improving its performance in the presence of such worst-case perception uncertainties.