ReinSeed: Reinforcement Fuzz Testing With Multiphase Seed Optimization for Autonomous Driving Systems

Ensuring the safety of autonomous driving systems (ADSs) is essential, which requires effective testing methods to enhance system robustness. Fuzz testing (FT) is a widely used technique for uncovering software faults by generating test cases that trigger unexpected system behaviors. However, traditional FT in ADS suffers from significant limitations, including inefficient seed selection, low test case relevance, and inadequate exploration of diverse failure‐inducing driving scenarios. Random fuzzing often yields redundant or ineffective cases, limiting the detection of safety‐critical issues. To address these challenges, we propose ReinSeed, a reinforcement FT (RFT) framework that integrates three key phases: prefuzzing seed optimization, reinforcement learning (RL)–based scenario generation, and postfuzzing seed prioritization. We introduce a scenario complexity index to prioritize initial seeds before fuzzing. During fuzzing, we model the process as a Markov decision process (MDP) and apply Q ‐learning to generate scenarios with effective fuzzing action variations guided by driving behaviors, including undesired behaviors and trajectory coverage. To further improve testing effectiveness, we present a postfuzzing prioritization strategy that ranks fuzzed scenarios based on risk energy by incorporating control constraint violation analysis, safety‐critical events, and risk‐driven trajectory. Experimental results demonstrate that the unified framework—ReinSeed—significantly improves the detection of undesired behaviors, outperforming baseline methods across maps of varying complexity. Furthermore, the multiphase seed optimization showcases distinct contributions of scenario complexity, behavior‐guided fuzzing, and risk energy in enhancing both the efficiency and effectiveness of discovering critical behaviors in ADS.