Federated deep reinforcement learning-based edge collaborative caching strategy in space-air-ground integrated network

To address the problem of limited network coverage in remote areas, combining space-air-ground integrated network with mobile edge computing could provide low-latency and high-reliability transmissions for user requests in these areas, as well as timely caching services. Considering the dynamic change of the topology of the space-air-ground integrated network and the content popularity being constantly updated, a network architecture of space-air-ground integrated edge collaborative caching was proposed first. Then, the cache replacement problem for edge servers was modeled as a Markov decision process. Finally, a federated discrete soft actor-critic (FDSAC) algorithm was proposed, with the core idea of integrating a weighted attention mechanism into the federated learning framework and incorporating a bidirectional long short-term memory network into the DSAC model. With the reconfigured reward function as the optimization objective, the optimal cache replacement policy was learned by maximizing the expectation of negative long-term rewards. Simulation results show that compared with other algorithm, the proposed algorithm can improve the cache hit rate of user requests by 18% and reduce the access latency of content by 25% while protecting user privacy.