Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization
Despite the rapid advancements of large language models (LLMs), LLM serving systems remain memory-intensive and costly. The key-value (KV) cache, which stores KV tensors during autoregressive decoding, is crucial for enabling low-latency, high-through
Authors
- Jiantong Jiang (ORCID: https://orcid.org/0000-0002-1624-3969)
- Rui Zhang (ORCID: https://orcid.org/0000-0002-1055-7609)
- Feng Liu (ORCID: https://orcid.org/0000-0002-5005-9129)
- Pei‐Yu Yang (ORCID: https://orcid.org/0000-0002-2406-4055)
Institutions
- The University of Melbourne (AU)
- Huazhong University of Science and Technology (CN)
Publication Details
- Published
- 2025-10-14
- DOI
- https://doi.org/10.36227/techrxiv.176046306.66521015/v1
- Citations
- 2
- Primary Topic
- Network Packet Processing and Optimization
- Type
- article
- Field-Weighted Citation Impact
- 4.54