Deep Reinforcement Learning for Joint UAV Trajectory and Communication Design in Cache-Enabled Cellular Networks

Unmanned Aerial Vehicles (UAVs) are now widely used in communication networks. They help in ‎delivering data in areas where the demand is high. This paper studies how UAVs work with cellular networks to provide better content transmission. The main goal is to reduce the time users wait to get the ‎content they need. The researchers suggest using edge caching with UAVs. This means UAVs store ‎popular data before users request it. The UAVs move based on an optimized path to deliver data efficiently. We also adjust transmission power. This reduces delays and improves the user experience. The ‎challenge is that users request data randomly. UAVs move dynamically, which adds uncertainty. Solving this problem with normal optimization methods is difficult. Instead, we use deep reinforcement ‎learning (DRL). We model the problem as a game where UAVs and a base station act as agents. These ‎agents observe the environment and make decisions accordingly. The paper introduces a new method ‎based on Proximal Policy Optimization (PPO). It is called Dual-Clip PPO. This method helps UAVs ‎explore the environment efficiently. It also ensures that actions are optimal over time. A new reward system ‎is introduced to guide UAV movement. The base station agent gets rewards from the environment, while ‎UAVs receive an extra reward when they explore new areas. Simulations show that this new approach ‎works better than existing methods. The proposed model reduces the time needed for users to receive ‎content. It also performs better than standard PPO-based learning methods. This paper concludes that ‎combining UAVs with caching and DRL improves communication networks. The method allows UAVs ‎to move sensibly, place content efficiently, and adjust transmission power.