CompactifAI: Extreme Compression of Large Language Models using Quantum-Inspired Tensor Networks

Large Language Models (LLMs) like ChatGPT and LlaMA offer immense opportunities but face challenges due to their vast size, leading to high training/inference costs and energy demands.Traditional compression methods focus on reducing neurons or precision.We introduce CompactifAI, a novel LLM compression using quantum-inspired Tensor Networks to compress the model's correlation space.Testing on LlaMA-2 7B showed a 93% memory and 70% parameter reduction, with minimal accuracy loss (2-3%) and significant training (50%) and inference (25%) speedups.