Generating evasive payloads for assessing Web Application Firewalls with Reinforcement Learning and Pre-trained Language Models

Web Application Firewalls (WAFs) serve as a critical defense mechanism against various web-based attacks such as SQL Injection (SQLi), Cross-Site Scripting (XSS), Server-Side Request Forgery (SSRF), Remote Code Execution (RCE), and NoSQL Injection. However, modern adversaries often craft evasive and obfuscated payloads capable of bypassing traditional WAF rules. To effectively assess and challenge the robustness of WAFs, we propose DEG-WAF, a Deep Evasion Generation framework that leverages Large Language Models (LLM) in conjunction with Reinforcement Learning (RL) to generate evasive payloads against WAFs. The system consists of four core components: a payload generation agent based on a pre-trained LLM (OPT-125M), a reward model that approximates WAF behavior, a grammar-based sampling agent that ensures syntactic validity, and an RL agent trained with either Proximal Policy Optimization (PPO) or Advantage Actor-Critic (A2C) to fine-tune generation strategies. Experimental evaluations on real-world WAFs, including ModSecurity and SafeLine, demonstrate that the A2C-based model significantly outperforms baseline LLMs—achieving a bypass success rate of 80.16% on SQLi and 74.70% on NoSQLi for ModSecurity, and 97.8% on RCE for SafeLine. These results underscore the potential of our LLM-RL framework to serve as a robust foundation for evaluating and enhancing the resilience of WAF systems under adversarial conditions.