Collaborative Edge Caching for Scalable Videos: A Two-Tier Federated Reinforcement Learning Method

The growing demand for video streaming services poses significant challenges in maintaining user experience while collaborative edge caching combined with scalable video coding offers a promising solution. In this paper, we formulate the challenge of optimizing collaborative edge caching and quality selection for scalable videos as an optimization problem, which we solve by federated reinforcement learning. However, how to handle heterogeneous user preferences across edge servers and coordinate different optimization objectives of policy and value network is a challenge. Therefore, we design a two-tier federated reinforcement learning method, utilizing distinct roles played by the policy network and value network, and accordingly applying different federated learning strategies for their training. Extensive experiments demonstrate the effectiveness of our approach.