Hybrid ReRAM-NMC & SRAM-CiM Matrix Multiplication for Large Language Model

This paper proposes a novel design called hybrid ReRAM-Near-Memory Computing (NMC) & SRAM-Computation-in-Memory (CiM) Matrix Multiplication (MM) for Large Language Model (LLM). Non-volatile memory ReRAM introduces device error (i.e. conductance shift) due to its nonidealities. To enable the deployment of LLM on ReRAM-based CiM, the proposed design enhances model's robustness by separating a small percentage of activations to SRAM-CiM and placing the remaining activations on ReRAM-NMC. Furthermore, a corresponding hardware architecture design is proposed to demonstrate the possible MAC circuit and hybrid-core pipeline. This paper demonstrates that the perplexity increases of Llama27B utilizing 6-bit precision quantization for both activations and weights are smaller than 2 for both gaussian error and shift error with the capacity percentage of SRAM less than 1%.