Fresh Caching of Dynamic Contents Using Restless Multi-Armed Bandits Over Wireless Access

We consider a dynamic content fetching, caching and delivery problem wherein the contents get updated at a central server, and local copies of a subset of the contents are cached at an edge cache associated with a base station (BS). Upon being requested, the BS can fetch the requested content from the central server and serve or can serve the locally cached version or can deny service. Fetching a content incurs a fixed fetching cost, serving a cached version incurs an ageing cost proportional to the age-of-version (AoV) of the content, and denying service incurs a fixed missing cost. Furthermore, due to unreliability of the channel between the BS and the users, a content delivery can fail, again resulting in a missing cost. We formulate an optimal content fetching, caching and delivery problem to minimize the average cost subject to the cache capacity constraint. This problem belongs to the class of continuous time restless multi-armed bandit (RMAB) problems with state dependent feasible action sets. We show that the single content problem is indexable, derive its Whittle indices and provide a Whittle index based policy for the original multi-content problem. Finally, we numerically evaluate the performance of the proposed policy and compare it to the existing works. We demonstrate that the proposed policy substantially outperforms the existing ones and, performance-wise, is very close to the optimal policy.