CNN Accelerator Adapted to Quasi Structured Pruning and Dense Mode Using FPGA

The multiplication-accumulation (MAC) in convolutional layers makes them computationally expensive, and these layers account for $\mathbf{9 0 \%}$ of the total computation. Several researchers have taken advantage of pruning the weights and activations to overcome high computation bandwidth. These techniques are divided into two categories: 1) unstructured pruning of the weights can achieve heavy pruning. 2) Structured pruning by the specified pattern prunes the weights and regularizes both computations and memory access. We propose Quasi Structured Pruning (QSP) that profits from the high pruning ratio of unstructured pruning. The load balancing property in structured pruning has also been included in the QSP scheme. Implementation results of our accelerator using VGG16 on a Xilinx XC7Z100 indicate $616.94 \mathrm{GOP} / \mathrm{s}$ and 1437.7 GOP/s at just 7.8 watts power consumption for dense and sparse mode, respectively.