How to Benchmark Post-Quantum Accelerators

With the recent standardisation of several post-quantum cryptographic (PQC) algorithms, there is an increased focus within the hardware research community to investigate efficient accelerator designs. However, it is difficult to determine whether the reported results are directly comparable due to differences in hardware prototyping platforms, integration contexts, and evaluation methodologies. This creates a gap between reported performance claims and their interpretability across works. In this paper, we survey the reported evaluation metrics across 50 PQC accelerators. We identify a reporting asymmetry across the literature in the evaluations of cryptographic hardware, limiting fair comparison and obscuring system-level trade-offs. To resolve this, we design a homogeneous, FPGA-based benchmarking framework to appropriately evaluate cryptographic hardware. We replicate five recent accelerators as a case study in the same RISC-V system-on-chip environment to fairly compare their design approaches and compromises.