Compressing and Accelerating Sparse CNNs Using Sign-Reserved Toeplitz Filters and Input Activation Density-aware Dataflow

Exploiting the sparsity in convolutional neural networks is crucial to accelerate computing and reduce energy consumption. Unstructured sparsity, benefiting from its flexibility to accommodate arbitrary sparse patterns, generally achieves higher accuracy, but it often introduces irregularity in convolutional operations, which complicates the control logic and undermines the benefits of sparsification. Structured sparsity alleviates these problems but sacrifices its application flexibility, which leads to lower accuracy. In this article, we propose TSCNN, an algorithm-hardware co-design solution that aims to compress and accelerate sparse CNNs while balancing both adaptability to sparsity and computational efficiency. In terms of algorithm, TSCNN adopts pruned filters compressed with sign-reserved Toeplitz matrix format (Tfilters), which systematically enhances the regularity of data reuse and flexibly reduces network parameters by 44%-86% while maintaining accuracy. In terms of hardware, TSCNN accelerator adapts to the structure of Tfilters and utilizes density-aware dataflows to support input activations with large sparsity variation, further optimizing the computational efficiency. Experiments show that TSCNN outperforms a dense CNN accelerator, sparse CNN accelerators SCNN and CSCNN, achieving 5.31× , 2.46× and 1.53× speedup and reducing energy consumption by 80.68%, 69.13% and 53.52%, respectively.