LiM Pruner: Efficient Pruning for Large Language-in-Memory Models

Large Language Models (LLMs) have revolutionized the fields of language understanding and generation, positioning themselves as transformative tools across a wide array of applications as general-purpose task solvers. However, the immense size of LLMs introduces significant challenges for deployment on traditional computing systems, largely due to the inherent von Neumann bottleneck. Recently, Computein-Memory (CiM) architectures have emerged as a groundbreaking solution to accelerate Deep Learning (DL) models, seamlessly integrating computation within the memory fabric to overcome the memory wall bottleneck. While there has been recent progress in mapping generative AIs and LLMs to CiM architectures, the deployment of these models with hundreds of billions of parameters remains very challenging, particularly for edge computing environments. To address the specific challenges of deploying LLMs in memory-centric architectures, we coin the term Language-in-Memory (LiM) to describe a new class of models specifically optimized for such systems. In this context, we present, for the first time, LiM Pruner, a suite of LiM compression techniques specifically designed to induce sparsity in pretrained LLMs. Additionally, we investigate data-path architectures to enhance support for pruned LiM models by addressing potential dislocation challenges. Our comprehensive evaluation of the LiM Pruner on the LLaMA-2 model demonstrates its superior performance compared to traditional pruning techniques.