The Cache Oracle: Preemptive Request Dispatching for I/O‐Asymmetric Serverless Databases

ABSTRACT In hyperscale, multi‐tenant serverless databases, the substantial performance disparity between cache hits (microseconds, CPU‐bound) and cache misses (milliseconds, I/O‐bound) leads to severe interference among tenants. Traditional reactive resource management models, which respond only after an expensive cache miss occurs, fail to prevent I/O‐intensive noisy neighbors from degrading the latency experienced by well‐behaved tenants. This paper proposes a novel preemptive isolation approach: the Predictive Cache‐Aware Dispatcher (PCAD). Embedded within the database proxy layer, PCAD leverages a real‐time machine learning model—the Cache Oracle—to predict the cache state of each request prior to dispatch. This foresight enables two key optimizations: (1) preemptive routing of likely cache misses to a dedicated, concurrency‐limited I/O worker pool, thereby protecting the CPU‐optimized fast path from contention, and (2) speculative prefetching to hide disk access latency for predicted misses. Using production traces from a large‐scale multi‐tenant cloud database, we demonstrate that the Cache Oracle achieves over 91% F1‐score in cache miss prediction. Simulations show that PCAD effectively eliminates cross‐tenant interference caused by I/O bursts, sustaining stable sub‐millisecond P99 latency, while speculative prefetching reduces perceived cache miss latency by over 90%. This preemptive predict and prevent approach marks a significant advancement toward more robust and high‐performance autonomous database systems.