TARDIS: A GPU-Centric KV Cache Service for Efficient LLM Inference

Key-value (KV) cache is a crucial optimization for large language model (LLM) serving, particularly in long-context inference scenarios. While existing KV stores suffer from a fundamental mismatch between the CPU-centric KV cache storage and the GPU-accelerated computation: (1) the low-parallelism CPUs cannot satisfy the fine-grained, highly parallel I/O demand, and (2) the synchronization between CPUs and GPUs introduced by layerwise asynchronous KV operations prevents the adoption of low-level optimizations (such as CUDA Graph).

Authors

Institutions

Publication Details

Published
2025-10-09
DOI
https://doi.org/10.1145/3725783.3764393
Primary Topic
Advanced Data Storage Technologies
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

TARDIS: A GPU-Centric KV Cache Service for Efficient LLM Inference

X. Wang, Yiming Zhang, Yifan Hu, Guangtao Xue et al.
Advanced Data Storage Technologies
article

TARDIS: A GPU-Centric KV Cache Service for Efficient LLM Inference

X. Wang, Yiming Zhang, Yifan Hu, Guangtao Xue, Lu Tang, Jiuming Yan, Hao Chen, Shi Qiu
article en

Abstract

Key-value (KV) cache is a crucial optimization for large language model (LLM) serving, particularly in long-context inference scenarios. While existing KV stores suffer from a fundamental mismatch between the CPU-centric KV cache storage and the GPU-accelerated computation: (1) the low-parallelism CPUs cannot satisfy the fine-grained, highly parallel I/O demand, and (2) the synchronization between CPUs and GPUs introduced by layerwise asynchronous KV operations prevents the adoption of low-level optimizations (such as CUDA Graph).

Shanghai Jiao Tong University (CN), Xiamen University (CN), Risun (China) (CN)
Openalex Percentile: Top 67%
Advanced Data Storage Technologies
DOISourceOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

TARDIS: A GPU-Centric KV Cache Service for Efficient LLM Inference — X. Wang, Yiming Zhang, et al. · (2025) | TGRS Research Map | TGRS