Cloud-edge-end collaborative caching and UAV-assisted offloading decision based on the fusion of deep reinforcement learning algorithms

The cloud-edge-end collaboration system provides a new impetus for the development of intelligent transportation. In order to optimize the quality of service for intelligent transportation system users and improve system resource utilization, A three-tier caching strategy for cloud-edge-end collaboration based on efficiency collaboration task popularity (CSEPCA) was proposed, which exploits server resource characteristics and performs fine-grained cache replacements based on real-time task popularity to address the challenges associated with balancing server cache space and cost. To achieve an optimal balance between server cache space and cost, the problem of determining the availability of server cache space is formulated as a constrained markov decision process (CMDP), and an enhanced deep reinforcement learning algorithm based on soft updating (AT-SAC) was designed to achieve multi-objective optimization of system latency, energy consumption, and resource depletion rate, with the aim of improving service response speed and enhancing user service quality. To address challenges in effectively serving vehicles in areas with weak communication signals from cloud-edge servers, UAV swarms were introduced to assist with vehicle task offloading computations. A comprehensive optimization algorithm (Co-DRL-P) was proposed, which integrates enhanced deep reinforcement Learning (ERDDPG) and improved particle swarm optimization (A-PSO) algorithms to optimize UAV trajectories and communication angles, aiming to deliver superior service quality to users. Finally, we evaluate the performance of the proposed scheme through comprehensive simulation experiments. Specifically, when the number of users is 30, the system latency of the proposed scheme is 17.9%, 11.5%, 2.6%, and 60.2% lower than baseline schemes such as DQN, DDPG, TD3, and collaborative randomized schemes, and the system energy consumption is reduced by 20.6%, 15.9%, 9.4%, and 129.9%. Notably, the overall system cost for drone-assisted user offloading is reduced by approximately 49.6% in areas with weak cloud server signals.