A Novel HDL Code Generator for Effectively Testing FPGA Logic Synthesis Compilers

Field Programmable Gate Array (FPGA) logic synthesis compilers (e.g., Vivado, Iverilog, Yosys, and Quartus) are widely applied in Electronic Design Automation (EDA), such as the development of FPGA programs. However, defects (e.g. incorrect synthesis) in logic synthesis compilers may lead to unexpected behaviors in target applications, posing security risks. Therefore, it is crucial to thoroughly test logic synthesis compilers to eliminate such defects. Despite several Hardware Design Language (HDL) code generators (e.g., Verismith) having been proposed to find defects in logic synthesis compilers, the effectiveness of these generators is still limited by the simple code generation strategy and the monogeneity of the generated HDL code. This paper proposes EvoHDL, a novel method to generate syntax-valid HDL code for comprehensively testing FPGA logic synthesis compilers. EvoHDL can generate more complex and diverse defect-triggering HDL code (e.g., Verilog, VHDL, and SystemVerilog) by leveraging the guidance of abstract syntax tree and the extensive function block libraries of cyber-physical systems. Extensive experiments show that the diversity and defect-triggering capability of HDL code generated by EvoHDL are significantly better than the state-of-the-art method (i.e., Verismith). In three months, EvoHDL has reported 20 new defects–many of which are deep and important; 16 of them have been confirmed.