TNSS:Two-Nibble Sparsity-Aware Stride-Decomposing Acceleration for Convolutional Neural Networks

Convolution Neural Networks (CNNs) are effective in image processing but suffer resource wastage due to sparsity in feature maps and weights, leading to inefficient computations. Compression of sparse data results in irregularity and challenges with feature map-weight matching for MAC operation due to different convolution strides. To address the above problems, a Two-Nibble Sparsity-Aware Stride-Decomposing (TNSS) scheme is proposed in this paper to efficiently eliminate zero-value computations in non-unit stride scenarios, while simultaneously considering bit-level sparsity to enhance inference efficiency further. In TNSS, tensors are initially decomposed into several unit stride tensors. Following this, TNSS compresses the feature map using two-nibble representation, reducing data size and computation load. Experimental results show that TNSS achieves an average speedup of $8.2 \times$, and $1.36 \times$ on VGG16 and MobileNetV1 compared with conventional architecture and the recent accelerator StarSPA, respectively.