HNM-CIM: An Algorithm-Hardware Co-designed SRAM-based CIM for Transformer Acceleration Exploiting Hybrid N:M Sparsity

SRAM-based computing-in-memory (CIM) is an efficient technology for computing neural networks where matrix operations are dominated. However, leveraging sparsity in CIM presents challenges due to the crossbar architecture, which complicates the avoidance of zero element calculations. Previous CIM designs have demonstrated that sparsity can improve energy efficiency, but these approaches often lead to non-negligible accuracy loss or substantial hardware overhead. To address this challenge, we propose a hybrid N:M CIM (HNM-CIM), an algorithm-architecture co-design framework for accelerating Transformers. At the algorithm level, we propose a hybrid N:M pruning (HNMP), a method that combines structured and unstructured sparsity. This approach maintains regularity while preserving the random distributions of sparsity, thereby enhancing model sparsity with negligible accuracy loss and ensuring CIM compatibility. At the hardware level, we introduce a hybrid N:M sparse digital CIM (HNM-CIM) to support HNMP, which can accelerate Transformers with hybrid N:M sparsity patterns. Experimental results show that HNMP can reduce Transformer models by about 3.1× on model size with negligible accuracy loss. Compared with state-of-the-art references, HNM-CIM yields about 2.46× speed up and 1.43× area savings.