Comma: A Communication-Minimized Model-Architecture Framework for Efficient Convolution Acceleration

Convolutional neural networks (CNNs) are still indispensable nowadays, but they bring a dominantly large overhead of data communication. Current works mainly focus on prior off-chip or intuitive/heuristic access optimization and lack a theoretically holistic analysis. Therefore, this article proposes an overall on- and off-chip communication-minimized CNN acceleration framework. First, we derive the layer-wise off-chip communication analytic expressions (AEs) based on different data reuse strategies and give their lower bound (LB), reaching 6%–18% lower than that proposed in work <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">CLB</i>. For practical off-chip access, we achieve 22%–35% reduction compared with the adaptive access strategy of work <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Sense</i> and 11% with the dataflow selection approach of work <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Versa</i>. Second, on-chip communication AEs and LB are derived within the tile range, and overall on- and off-chip access analysis is conducted to obtain the unified energy LB (ELB). Guided by our theoretical model, we design window-primitive (WP) dataflow and an energy-efficient systolic-cross-line (SCL) CNN accelerator. Finally, a holistic framework, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Comma</i>, is proposed to streamline the workflow of model-architecture co-design. SCL is implemented in 65-nm technology and achieves <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$4.2\times $ </tex-math></inline-formula> pJ/Multiply and ACcumulation (MAC) energy reduction at <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1.4\times $ </tex-math></inline-formula> less on-chip memory area compared with <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Eyeriss</i>, and 1.4~<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1.9\times $ </tex-math></inline-formula> reduction at 3~<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$4\times $ </tex-math></inline-formula> less area compared with work <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">CLB</i>; as for systolic architecture, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Sense</i>, it achieves 29%–44% energy reduction with 4% area overhead. For the near memory processing (NMP) situation, we reduce around <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2\times $ </tex-math></inline-formula> access energy compared with the systolic NMP architecture.