QSLR: Post-Training Compression via Quantized Sparse and Low-Rank Factorization

The growing scale and complexity of transformer-based foundation models have led to significant challenges in deploying them efficiently, particularly on resource-constrained environments. While recent methods such as post-training pruning or sparse + low-rank decomposition reduce model size without retraining, they still rely on full-precision weights, limiting gains in memory bandwidth and latency. In this paper, we propose QSLR, a unified post-training quantization (PTQ) framework that combines outlier-aware pruning via sparse and low-rank decomposition with component-wise Hessian-aware quantization. Each decomposed component (sparse matrix, low-rank left factor, low-rank right factor) is quantized independently using projected Hessians, with an efficient approximation to eliminate redundant Hessian computations. We further optimize the quantization parameters via Hessian-weighted grid search to minimize second-order quantization loss. Experiments on LLaMA2-7B and ViT-Base demonstrate that QSLR achieves up to 5×model compression with minimal accuracy degradation, consistently outperforming state-of-the-art pruning or quantization methods.