A 28 nm 128-kb Exponent- and Mantissa-Computation-In-Memory Dual-Macro for Floating-Point and INT CNNs

Floating-point computing-in-memory (FP-CIM) has emerged as a promising approach to enhance the energy efficiency of cloud and edge artificial intelligence (AI) devices. However, recent advancements in large language models (LLMs) and complex neural networks demand efficient support for both FP and integer (INT) multiplication-and-accumulation (MAC) operations to achieve high performance. This work presents a 28 nm 128 kb exponent-computing-in-memory (ECIM) and mantissa-computing-in-memory (MCIM) dual-macro, which effectively supports both true FP and INT MAC operations. The contributions of this work include: 1) a dual-macro architecture enabling both INT and true FP MAC operations; 2) an ECIM unit for digital exponent (EXP) addition and analog INT MAC; 3) an EXP-mantissa alignment (EMA) unit with early exit capability for outlier situations; and 4) an MCIM unit featuring two-level approximate adder trees and offline LUT-based retraining. The fabricated chip achieves 45.3 TFLOPS/W in energy efficiency for FP MAC with over 91.44% hardware utilization, demonstrating significant performance and energy efficiency improvements compared to existing solutions.