Re-evaluating VERT: A Classification Framework for LLM-Generated System Verilog Assertions

This paper presents the first systematic failure analysis of LLM-generated SystemVerilog assertions, replicating the VERT fine-tuning pipeline on OpenTitan's AES, UART, and SPI modules. After validating 125 assertions through Cadence JasperGold, results reveal a 75–83 percentage point gap between VERT's reported 96.25% functional correctness and the 12–30% success rate observed under formal verification, exposing limitations on VERT’s evaluation methodology as well as limitations in experiment replication. To explain this gap, I introduce a 4×7 classification framework organizing assertions by verification outcome and root cause, identifying configuration-dependent logic and temporal operator confusion as the dominant failure modes that must be addressed. I also argue for increased transparency and open-sourcing of LLM training and evaluation materials as a lack of reproducible experiments can hinder progress towards an LLM assertion generation engine that can reliably serve production hardware verification.