High-Performance Gemmini-Based Matrix Multiplication Accelerator for Deep Learning Workloads

Transformer models have recently gained considerable attention in computer vision applications due to their capacity to capture relations among features, resulting in enhanced performance. Moreover, deep neural networks (DNNs) have been extensively researched due to their superior performance and applicability in several domains, including image classification, detection, and recognition. Matrix multiplications are crucial operations in Transformers and DNN, and computations such as weight stationary (WS) and output stationary (OS) are considered to meet the dataflow constraints. This work presents a systolic array (SA)-based general matrix multiplication (GEMM) architecture for Gemmini accelerators, enabling the performance of convolutions in neural networks and self-attention in Transformers. First, this work proposes a multiplier-optimized SA (MOSA) by integrating a novel multiplier. The multiplier is designed using a single-stage stacking-based 3:2 and 4:2 compressors. The proposed MOSA achieves significant improvements, with area savings ranging from 12% to 92% for 8-bit designs and 87%–89% for 16-bit designs across <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$8 \times 8$ </tex-math></inline-formula>, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$16 \times 16$ </tex-math></inline-formula>, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$32 \times 32$ </tex-math></inline-formula>, and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$64 \times 64$ </tex-math></inline-formula> SAs. The proposed multiplier is employed to present a high-performance Gemmini SA for WS and OS dataflow; further modifications to the processing element (PE) are presented to achieve better performance over the existing Gemmini accelerator. In particular, the proposed WS PE achieves power-delay product (PDP) savings of 56%–59% and area-delay product (ADP) reductions of 45%–51%, while the OS PE shows PDP savings of 10%–46% and ADP reductions of 6%–42%. The analysis is further extended to both convolutional neural networks (CNNs) and Transformers by integrating the proposed SA into complete inference pipelines, including AlexNet, MobileNetV2, and ResNet-50 for CNNs, and the self-attention mechanism of TinyBERT for Transformers. The MOSA-32 design delivers a GOPS improvement of 12%–91% across different CNN models and up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$5.24\times $ </tex-math></inline-formula> for Transformer workloads. Similarly, MOSA-64 achieves up to 98% improvements across CNNs and up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$6.14\times $ </tex-math></inline-formula> gains for Transformers. These improvements validate the effectiveness of the proposed energy-efficient PE design and its seamless integration into Gemmini SA, making the proposed architecture suitable for next-generation AI and Transformer accelerators for edge computing applications.