Exploiting N:M Sparsity in Quantized-Folded ResNets: Signed Multicoat Supermasks and Iterative Pruning-Quantization

Neural networks provide excellent performance, but operating them under limited resources remains challenging due to their high computational cost. Therefore, pruning techniques to reduce the computational cost of neural networks are gaining attention. In particular, N:M sparsity, a structured pruning method, offers an excellent trade-off between sparsity and image recognition accuracy. More hardware products, such as GPUs and AI accelerators, support N:M sparsity. In this paper, we apply N:M sparsity and quantization to Folded ResNets, where weights are shared for some iterative blocks, to find a good tradeoff between model size and image recognition accuracy. First, we investigate the performance of an algorithm that combines two state-of-the-art methods in N:M pruning and quantization, SRSTE and LSQ, respectively. We also investigate an N:M extension of the Signed Multicoated Supermasks (SMS) algorithm, which is based on the Strong Lottery Ticket Hypothesis (SLTH). Although the SMS-based method outperforms the LSQ-based method in unstructured pruning, our experiments show that the SMS-based method is inferior in N:M sparsity. This accuracy degradation is caused by N:M structured sparsity constraint narrowing the search space of the strong lottery ticket’s subnetworks. To address this degradation, we propose adding noise to the initial weights to improve accuracy. This approach consistently improves the accuracy of the N:M SMS across a variety of sparsity levels, while ensuring that the parameter size increase remains minimal.