Cooperative Edge Caching with Multi-Agent Reinforcement Learning Using Consensus Updates

In modern wireless networks, efficient caching strategies are critical for reducing user access delays and optimizing network performance. However, the transmission bottleneck caused by frequent information exchanges between base stations (BSs) and the cloud server in the Centralized Training and Decentralized Execution (CTDE) framework poses a significant challenge. To address this issue, we propose a Distributed Training and Decentralized Execution (DTDE) based cooperative caching method that eliminates the need for in-formation exchange between BSs and the cloud server. We further introduce Single Neighbor DTDE (SN-DTDE) model to reduce information exchange among BSs. Additionally, we apply consensus update methods to improve DTDE and SN-DTDE model performance. The parameter sharing method shares Critic network parameters among neighboring BSs, which aligns their objectives but increases communication overhead. To mitigate this, we propose a value-sharing method that only shares TD errors, significantly reducing information exchange. Our results show that the value-sharing method effectively enhances model performance with minimal communication overhead, making it suitable for practical applications.