SpiderSense: Lightweight Last-Level Cache Management via Time Period Tagging for LLC-Critical Workloads

Multi-tenant clouds enhance resource sharing among Virtual Machines (VMs) to boost overall utilization and reduce power consumption. However, this also introduces interference among workloads from different tenants and impedes VM performance isolation. In this paper, we first demonstrate that the last-level cache (LLC) in CPUs, which is inherently shared by all VMs on the same physical machine, becomes a significant contending resource for LLC-critical workloads, leading to notable performance imbalances under the default hardware caching strategy. Although recent studies on LLC scheduling have progressed, they often require detailed profiling of user workloads or rely on hyperparameter tuning, limiting their applicability to private clusters or specific scenarios. We propose SpiderSense, a software-initiated LLC partitioner for managing Virtual Machine Monitors (VMM), to address these limitations. SpiderSense leverages modern yet off-the-shelf server CPU features to adaptively orchestrate LLC allocation among running black-boxed user VMs. SpiderSense dynamically samples VMs and calculates their fair share of LLC to allocate them while fully improving performance isolation among VMs. We experiment with SpiderSense using typical LLC-critical workloads, representative of the types of applications that stress LLC performance, such as Memcached and Llama. Our results show that SpiderSense improves performance by up to 40% in numerous colocation scenarios compared to current solutions.