OHDRL-Based Energy Consumption Optimization for Joint Content Fetching and Trajectory Design of UAVs

In this study, we investigate minimization of energy consumption in multi-UAV assisted networks. We formulate an energy minimization optimization problem with UAV trajectory design, content fetching, power allocation and content placement constraints. The problem is a mixed integer nonlinear programming (MINLP); therefore, we convert the formulated problem into semi-Markov decision process (SMDP). To tackle this SMDP optimization challenge, we introduce an option-based hierarchical deep reinforcement learning (OHDRL) approach. We designate UAV trajectory planning and power allocation as the low level action space, and content placement and content fetching as the high level option space. Through simulations, we demonstrate the effectiveness of the proposed OHDRL method.