STAPC: A Sparse Training Accelerator for Efficient On-Device DNN Learning via Position Constraints

Leveraging sparsity to eliminate redundant computation and storage overhead is essential for enhancing the efficiency of on-device deep neural network (DNN) learning. However, due to a lack of assumptions about non-zero positions, existing sparse methods incur high costs for zero-position identification and allocation, making it difficult to achieve ideal acceleration. This paper demonstrates that knowing non-zero position constraints in advance during training can bypass these sparse processing overheads. We explore non-zero position constraints among operands for three typical activation functions in edge scenarios and propose: (1) a hardware-friendly sparse training algorithm to skip redundant gradient computations, enhancing training efficiency; and (2) a high-efficiency sparse training accelerator, STAPC, that estimates non-zero gradient positions, allowing costly sparse processing to be masked in parallel to reduce energy consumption. Compared to the baseline and other sparse training methods, the proposed method achieves energy efficiency gains of 2.2x, 1.38x, and 1.46x, respectively.