A Mixed-Precision Transformer Accelerator With Vector Tiling Systolic Array for License Plate Recognition in Unconstrained Scenarios

Power efficiency for license plate recognition (LPR) under unconstrained scenarios is a crucial factor in many edge-based real-world applications, e.g., autonomous vehicles whose power budget is limited. While a bulk of prior works have explored LPR approaches for unconstrained situations on CPU and GPU servers, these methods result in huge power dissipation, and are ineffective in challenging scenes. In this work, we present a mixed-precision (MP) Transformer hardware architecture to meet the requirements of power efficiency and satisfactory LPR accuracy in unconstrained scenarios, dedicated to implementing power-efficient edge accelerators for difficult LPR tasks. Firstly, MP-LPR Transformer model is proposed, where novel mixed-precision quantization and non-linear approximation techniques are tailored for low bit-width inference. Secondly, vector tiling systolic array Transformer architecture (VTSATA) is proposed with unique vector tiling systolic array (VTSA) and vector ALU (VALU) designs, where VTSA is used to accelerate matrix multiplications, and VALU is used to process non-linear operations. Finally, a FPGA accelerator prototype based on our approach is developed. Experimental results demonstrate that our hardware platform can reduce power consumption more than <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$20\times $ </tex-math></inline-formula> compared to GPU platforms, and for challenge subset on CCPD dataset, our method can further improve nearly 1% accuracy with respect to the state-of-the-art performance.