A Reinforcement Learning Method for UAV Delivery Scheduling Under Dynamic Pricing

The on-demand delivery service of Unmanned Aerial Vehicles (UAVs) has attracted attention and research worldwide. Under the on-demand delivery model, the UAV transports parcels between ports (e.g., a designated location for UAV to drop off/pick up parcels), to satisfy the rapid response to urgent demand. Due to the complexity of large-scale routing problems and response time limitation, the general heuristic algorithms may obtain locally optimal scheduling solutions, thus reducing the delivery utility. Therefore, we propose a UAV route scheduling scheme based on a two-step hybrid heuristic algorithm. In particular, the biogeography-based optimization algorithm is applied to obtain the optimal set of UAV tasks at the first step. At the second step, the approximate dynamic programming is proposed to schedule task routes, to obtain the global optimal solution. Besides, the price function based on delivery status and demand characteristics is trained through the advantage actor critic algorithm to determine the optimal delivery price, so that the flexibility of UAV scheduling is increased. Based on simulation experiments, our scheme outperforms other solutions on increasing delivery utility, and can provide the optimal delivery price based on customer characteristics.