Cache Partitioning for Mitigating Timing Side-Channel Attacks in LLM Serving Systems

The continuous development of Large Language Models (LLMs) has led to a significant demand for improving their inference efficiency. However, recent studies indicate that shared Key-Value (KV) cache, a widely used optimization techniques, can be exploited as a side-channel to recover user's prompts in multi-users settings, which is confirmed to be widespread across various LLM inference services, posing threats to user's privacy. This paper proposes a user-boundary-based cache partition mechanism to mitigate the above prompt leakage induced by shared caches. Our design partitions the KV cache into different prefix trees by user identities, preventing attackers from detecting cache hits and recovering shared prompt prefixes in the caches. The experimental results shows that cache partitioning mechanism can eliminate the timing difference, and the system's overhead remains within an acceptable range (6.44% in Time- To-First- Token and 3.06% in throughput) across different models and scenarios.