A Workload-Balance-Aware Accelerator Enabling Dense-to-Arbitrary-Sparse Neural Networks

Deep neural networks (DNNs) have proved their great potential over various perceptual and cognitive tasks with the cost of ever-growing storage capacity and computation complexity. Sparse representations in neural networks have emerged as a compelling method to achieve substantial reductions in computational overhead and energy consumption. However, the introduction of sparsity presents challenges such as irregular memory accesses and wasted computation cycles. Traditional methods have attempted to address these challenges with varied success, unfortunately, often involving complex hardware designs or sacrificing sparsity’s benefits. In this article, we propose a software-hardware codesign solution, comprising an offline workload balancing encoding algorithm toward arbitrary sparsity in DNNs and a dedicated Processing Element array–based accelerator with a lightweight switch network. Extensive experiments are conducted to demonstrate the conclusion that our proposal is feasible to support various network structures and a wide range of sparsity ratios. With the encoding algorithm, a 1.16×–2.61× acceleration is achieved compared to the baseline. The system-on-chip measures 7.9 mm 2 , achieving an energy efficiency of 0.7 TOPS/W (dense), 2.1 TOPS/W (at 75% sparsity), and 10.3 TOPS/W (at 99.9% sparsity) at 0.9 V and 1,066-MHz clock frequency.