Improving cache-enabled D2D communications using actor–critic networks over licensed and unlicensed spectrum

Cache-enabled Device-to-Device (D2D) communications is an effective way to improve data sharing. User Equipment (UE)-level caching holds the potential to reduce the data traffic burden on the core network. Licensed spectrum is utilized for D2D communications, but due to spectrum scarcity, exploring unlicensed spectrum is essential to enhance network capacity. In this paper, we propose caching at the UE-level and exploit both licensed and unlicensed spectrum for optimizing throughput. First, we propose a reinforcement learning-based data caching scheme leveraging an actor–critic network to improve cache-enabled D2D communications. Besides, licensed and unlicensed spectrum are devised for D2D communications considering interference from existing cellular and Wi-Fi users. A duty cycle-based unlicensed spectrum access algorithm is employed, guaranteeing the Signal-to-Interference and Noise Ratio (SINR) required by the users. The unlicensed spectrum is prone to data packets collisions. Therefore, Request-to-Send/Clear-to-Send (RTS/CTS) mechanism is utilized in conjunction with Carrier Sense Multiple Access with Collision Avoidance (CSMA/CA) to alleviate both the interference and packets collision problems of the unlicensed spectrum. Extensive simulations are performed to analyze the performance gain of our proposed scheme compared to the benchmarks under different network scenarios. The obtained results demonstrate that our proposed scheme possesses the potential to optimize network performance.