S4T-GPTQ: A Space-for-Time Strategy For Optimizing GPTQ 4-bit Quantization

Frequent global memory accesses and computational complexity often become significant performance bottlenecks in large language model (LLM) inference. To address this challenge, this paper proposes a space-for-time strategy for opti-mizing GPTQ 4-bit quantization, namely S4T-GPTQ. The S4T-GPTQ leverages shared memory blocks to store intermediate results, significantly reducing the number of global memory accesses. Experimental results show that S4T-GPTQ improves the inference throughput and reduces the inference latency of the GPTQ 4-bit quantization under the vLLM framework. These enhancements provide an efficient and practical solution for deploying large language models (LLMs) in high-throughput, low-latency scenarios.