Adaptive Caching and Request Routing for Latency-Sensitive Serverless Applications
Cold-start overhead and uneven placement of warm function instances can amplify request latency in serverless deployments. This paper studies a compact joint policy, AJR-ARF, combining latency-aware routing with a decayed recency-frequency warm-instance cache. At each arrival, the dispatcher estimates network, queueing, execution, and initialization components; after dispatch, a node replaces the cached function with the largest popularity-adjusted idle age. The method does not require offline training or workload forecasts. A deterministic discrete-event simulator was executed on changing-popularity synthetic requests across six nodes. Across five random seeds, the proposed policy produced 298.6 ms mean per-run 95th-percentile response latency, compared with 597.6 ms for nearest-node LRU, 392.3 ms for queue-aware LRU, and 302.8 ms for cache-aware LRU. The experiments also report cold-start frequency, SLO attainment, and sensitivity to cache capacity, traffic intensity, and popularity skew. The large reduction comes primarily from warm-state-aware routing, while adaptive eviction provides a smaller incremental change under this model.
Authors
- Iris Jansen (ORCID: https://orcid.org/0000-0003-4542-6155)
- Daan Vermeer
- Sophie van Leeuwen
Institutions
- Utrecht University (NL)
- Eindhoven University of Technology (NL)
Publication Details
- Journal
- Academic Journal of Applied Sciences
- Published
- 2026-11-16
- DOI
- https://doi.org/10.54097/nnt69h42
- Primary Topic
- Cloud Computing and Resource Management
- Type
- article
- Field-Weighted Citation Impact
- 0.00