HiTEA-3D: A Design Framework for High-Throughput and Energy-Aware 3D CNN Acceleration on FPGA

This paper introduces HiTEA-3D, a design framework for High-Throughput Energy-Aware acceleration of 3D Convolutional Neural Networks (CNNs) in resource-constrained environments such as Field Programmable Gate Arrays (FPGA), through mixed-precision quantization, dynamic tiling, and multiple data fetching modes. The HiTEA-3D framework accepts a PyTorch-described model and FPGA specifications as input and generates a synthesizable Verilog code for implementation. The resulting FPGA based design employs a 3D systolic array architecture to maximize the data reuse across spatio-temporal dimensions. The size of the 3D systolic array is determined by minimizing latency while effectively utilizing the available FPGA resources. The basic Processing Element (PE) block used for computing is designed to support variable word-lengths for both the input feature maps and the kernel blocks. The proposed framework has been evaluated on three popular 3D CNN models, C3D, I3D, and R(2+1)D, and implemented on Xilinx Virtex-7 and Zynq UltraScale FPGAs. The experimental results have shown that the hardware accelerators generated through the HiTEA-3D framework deliver improved performance, achieving an average increase of 50% in throughput and a 33% reduction in energy consumption across all tested networks. The hardware architecture is parameterized and can be ported to various implementation platforms. The designs can be quickly produced by specifying high-level specifications of CNN models and FPGA resources.