Scheduling Drone and Mobile Charger via Hybrid-Action Deep Reinforcement Learning

Recently, there has been a growing interest in using chargers to extend the operational longevity of UAVs (drones). In this paper, we explore a charger-assisted drone application where a drone observes points of interest while a mobile charger moves to recharge its battery. We focus on the route and charging schedule of the drone and mobile charger to maximize observation utility in the shortest possible time, while ensuring continuous drone operation. In our problem, the drone and mobile charger cooperate to complete a task. Their discrete-continuous hybrid actions pose a major computational challenge. To address this issue, we present a hybrid-action deep reinforcement learning framework, called HaDMC, which uses a typical policy learning algorithm to generate latent continuous actions. We specifically design and train an action decoder. It involves two pipelines to convert the latent continuous actions into the original hybrid actions for the drone and mobile charger to directly interact with environment. We incorporate a mutual learning scheme into model training, emphasizing collaboration over individual actions. By extensive numerical experiments, we evaluate HaDMC and compare it with state-of-the-art approaches. The experimental results demonstrate the effectiveness and efficiency of our solution.