RL–LLMfuzzer: reinforcement learning–guided dual–model differential fuzzing for compiler testing

Compiler fuzzing faces a fundamental dilemma: traditional rule-based generators easily fall into coverage bottlenecks due to rigid grammar constraints, while Large Language Models (LLMs), despite synthesizing semantically rich code, suffer from extremely low validity rates in their raw outputs, causing a severe waste of inference resources. To resolve this contradiction, this paper proposes RL-LLMfuzzer, a reinforcement learning and LLM-based differential fuzzing framework. The core innovation lies in its pioneering dual-model decoupled architecture: it utilizes a Multi-Armed Bandit (MAB) algorithm to drive a high-capacity prompt evolution model, dynamically optimizing high-level testing intents based on coverage feedback; subsequently, a local code generation model instantiates these intents. To overcome the inherent syntactic fragility and inference latency of LLMs, the framework integrates a lightweight heuristic repair module and a deterministic AST mutation engine, structurally amplifying valid seeds before differential evaluation. Extensive experiments on GCC and Clang demonstrate superior testing efficacy: verified across multiple independent 24-hour trials, RL-LLMfuzzer achieved a statistically significant 32.1% increase in average compiler backend line coverage in the C-language benchmark compared to the state-of-the-art LLM fuzzer Fuzz4All ( $$p < 0.001$$ ). The generated corpus significantly outperforms baseline tools like Csmith, YARPGen, and GrayC in multi-dimensional diversity. To date, RL-LLMfuzzer has successfully unearthed 28 officially confirmed vulnerabilities in GCC and LLVM/Clang, establishing a highly efficient and scalable paradigm for LLM-driven compiler validation.