A Novel Deep Reinforcement Learning Framework for Shared Autonomous Vehicles Pickup, Delivery, and Charging Problem

Shared Autonomous Vehicles (SAVs) are expected to continuously operate with Dynamic Wireless Charging (DWC), replacing most private vehicles in intelligent public transportation. Solving the SAV’s Pickup, Delivery, and Charging Problem (SPDCP) requires optimising route costs and stabilising the battery state of charge (SOC) of SAVs. However, existing heuristic and Deep Reinforcement Learning (DRL) algorithms cannot solve SPDCP, as they fail to handle heterogeneous nodes and adaptively access wireless charging based on battery levels. This paper, for the first time, proposes a heterogeneous DRL (HDRL) framework for solving SPDCP to balance multi-objective optimisation and SOC stability management. We develop a novel heterogeneous attention mechanism in HDRL to enhance the comprehension of node sequential constraints and SOC stability. To assist HDRL decision-making, a soft-constraint-based mask scheme directs strategies to prioritise pickup and delivery tasks at high SOC levels and incorporates more DWC nodes when SOC is low. In multi-scale tests, compared to the optimal results of the heuristic and other DRL algorithms with the given initial solutions, HDRL reduces the route cost by 8.6%, 6.0%, and 2.1% in the 20,50, and 100 scale tests, respectively. For battery level management, HDRL obtains better SOC variation control than other DRLs, 20-Node: 0.08, 50-Node: 0.22, 100-Node: 0.23. Furthermore, HDRL has demonstrated its adaptive control capabilities at various SOC levels, dynamic traffic scenarios, and robust cross-scale, cross-environment generalisation through outstanding verification results.