An 800-MHz 8.17-TOPS/W 0.63-TOPS/mm ² Memory-Utilization-Aware CNN Accelerator Featuring a Memory Stationary Dataflow

Increasing the on-chip memory utilization (OCMU) is crucial for an area-efficient deep neural network accelerator. We propose a memory stationary (MS) dataflow to ingeniously combine the input and output features in a single memory block in a cyclic manner, significantly increasing the OCMU. The MS dataflow also reduces the feature memory access by 78.0%. Furthermore, residual paths in a ResNet model require large feature buffering. We introduce layer-wise clipped-asymmetric residual distillation (LCARD) quantization, removing the residual paths with minimal accuracy degradation. It dynamically assigns different feature/weight bit-widths for different layers, further enhancing the OCMU by <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$3.2{\times }$ </tex-math></inline-formula> and throughput by <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$4.5{\times }$ </tex-math></inline-formula> from a fixed bit-width (FBW) approach. We also present an MS gating (MSG) to skip the ineffective channels that improve the OCMU by <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1.2{\times }$ </tex-math></inline-formula> and throughput by <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1.3{\times }$ </tex-math></inline-formula>. Fabricated in a 28-nm CMOS process, the proposed accelerator exhibits an 8.17-TOPS/W peak energy efficiency and a 0.63-TOPS/mm2 peak area efficiency at 800 MHz and 0.9 V while requiring only a 120 kB on-chip static random-access memory (SRAM) for the ResNet-50.