A 22nm 29.3TOPS/W End-to-End CIM-Utilization-Aware Accelerator with Reconfigurable 4D-CIM Mapping and Adaptive Feature Reuse for Diverse CNNs and Transformers

With the rapid evolution of neural networks (NNs), a variety of state-of-the-art models based on convolutional neural networks (CNNs) and transformers have emerged, which have been extensively applied in various fields such as computer vision (CV), natural language processing (NLP), and multimodal domains. Operators in CNNs and transformers exhibit significant diversity in dimensions, computational modes and data access patterns, as shown in Fig. 1. This diversity leads to distinct latency and power bottlenecks and requires flexible hardware architectures for efficient acceleration. However, existing NN accelerators, including conventional digital accelerators [1]–[3] and compute-in-memory-based (CIM-based) accelerators [4]–[8], are mainly dedicated to specific models, lacking the flexibility to efficiently support a wide range of models. First of all, the CIM arrays are generally designed as rigid structures with fixed parallelism, leading to under-utilization when mapping operators of varying dimensions onto them. Secondly, in certain layers of NNs, the feature footprint can be very large, especially in attention layers where the size of intermediate features grows quadratically with token length. The large footprint leads to intensive external memory access (EMA), bringing significant latency and power overhead. Data reuse is an effective strategy to reduce memory access however, existing accelerators only employ feature reuse In local registers [7] [8], which is insufficient to reduce EMA for operators of varying access patterns. Thirdly, existing architectures have largely neglected the abundant vector operators in different models, which can significantly contribute to end-to-end latency and power consumption in certain scenarios. For example, softmax operations in transformers occupy a large fraction of runtime with long sequence lengths [9].