Fast CPU RTL verification using FPGA accelerated co-emulation

As computer systems continue to grow in complexity their verification requirements grow dramatically placing significant strain on how to effectively and efficiently prove the hardware performs the correct set of operations under all possible scenarios. The current start of the art begins with a design verification (DV) plan to establish what needs to be tested, what stimulus to drive into the device under test (DUT), and how to measure the design is actually tested (coverage). Testing is typically done at the register transfer level (RTL) using simulators, such as Questa or Verilator which execute a few thousand instructions per second significantly limiting the scope of the tests. Specialty hardware costing millions of dollars exists which can run a few million instructions per second but this is only available to large chip developers such as Intel, Apple, and ARM. FPGAs can execute dozens of millions of instructions per second but lack the visibility to measure what is tested or efficiently debug when a problem is found. State-of-the-art verification runs the DUT in lockstep with a robustly tested reference model which compares all architecture state updates as each instruction commits. Any mismatches immediately halt the simulation indicating where and what caused the bug. Like any RTL simulation, lockstep is slow. This work develops an entirely new methodology for verification called co-emulation that leverages the speed of FPGAs using a reference model in lockstep to overcome the limitations of the FPGA. This work shows a significant speed-up greater than 35 times over RTL simulation using a low-cost FPGA and the robust commercial reference model ImperasDV from Synopsys. The device under test (DUT) in our setup is the CVW processor, known by its code named Wally. Because Wally is mature, the bugs found are difficult to discover using other testing paradigms, highlighting just how valuable a wide diversity of verification approaches is to the DV plan.