A 28nm 128-kb Exponent- and Mantissa-Computation-InMemory Dual-macro for Floating-point and INT CNNs

Floating point computation-in-memory (FP-CIM) has garnered significant attention in enhancing the energy efficiency of cloud and edge Al devices [1–3]. Recent advancements in large language models (LLM) and complex neural networks necessitate support for both floating point (FP) and integer (INT) multiplication-and-accumulation (MAC) operations to achieve superior performance and energy efficiency. Past FP-CIM implementations have utilized two primary methods for FP MAC operations: one involves utilizing extra exponent-mantissa alignment (EMA) and INT-FP quantization circuits, which necessitates offline data type conversion and isn’t considered true FP-CIM [1]. The other approach involves the use of additional exponent summation array and alignment circuits, enabling true FP-CIM but leading to increased area and energy consumption [2]. Efficiently implementing both true FP- and INTMAC operations can pose challenges for CIM macro. These challenges include: (a) the substantial area and energy overhead caused by extra exponent summation array (ESA) circuits when applied to INT MAC; (b) the significant power cost of readout circuits in previous analog or hybrid-domain CIM works; (c) the possibility of the aligned mantissa exceeding the maximum supported bit-precision of CIM size; and (d) the higher cost and power consumption of the larger adder tree in previous digital-domain CIM works, as shown in Fig. 1. This work presents a 28 nm 128kb exponent-computation-in-memory (ECIM) and mantissa-computation-in-memory (MCIM) dual-macro. The dual-macro can effectively support both true FP and INT MAC operations, offering the best energy efficiency (45.3 TFLOPS/W) in average case and the highest hardware utilization (>91.44%) for true-FP and INT configurations.