Spicing Up LLMs: The Role of PAPRICA Pruning in Linear Computation Coding

The main contributor to the computational cost of the inference phase of neural networks is the constant matrixvector multiplication. Linear computation coding has emerged as a novel approach for reducing the complexity of constant matrix-vector multiplications. In this paper we demonstrate that linear computation coding methods are compatible with pruning of large language models and reduce the computational cost of the inference phase beyond what pruning alone can achieve. We focus on the perplexity as a metric to identify columns suitable for pruning. Layers with lower importance are pruned in a systematic columnwise manner. This structured pruning approach directly benefits the linear computation coding algorithms for an additional gain. Applying fine-tuning to the pruned models leads up to a 4.5 -fold decrease of the computational cost while maintaining over 85 % of the zero-shot accuracy of several natural language tasks. We also discuss the trade-off between the computational cost and the performance of large language models.