Optimal Scheduling and Deep Reinforcement Learning for Multimodal Charging System via Unmanned Aerial Vehicles

Unmanned aerial vehicles (UAVs) exhibit considerable promise in the domain of package delivery, with an emphasis on efficiency and environmental sustainability. However, UAVs face challenges in delivering packages over expansive areas due to the limited battery capacity. In this paper, we explore the concept of UAVs hitching on vehicles as a means to conserve energy. To alleviate the delivery inefficiency caused by frequent hitching, we propose a new multimodal charging system to replenish the energy of the UAV while it delivers packages. This multimodal charging system is formulated as a two-layer model that considers inductively wireless power transfer (WPT) resources allocation and laser power scheduling to minimize the maximum completion time of the logistics system. In layer I, a limited number of vehicles are equipped with wireless charging devices, allowing UAVs to harvest energy while hitching vehicles. We model the charging vehicle allocation problem as an optimization problem that minimizes the maximum completion time of the system. In layer II, UAVs can harvest laser energy without the need for hitching a vehicle. The selection of which UAV to charge is structured as a Markov decision process, and a dueling deep Q-network (DQN) with prioritized experience replay is proposed to manage laser energy. Our results show that either the layer I or layer II algorithm alone improves the efficiency of UAV delivery. In addition, we verify that the two-layer algorithm can collaboratively optimize the maximum system completion time, which demonstrates the effectiveness of our proposed multimodal charging system.