Research on scenario-adaptive D4QN energy management with real-time control for HEVs

Reinforcement learning-based methods are gaining popularity in energy management strategy (EMS) for hybrid electric vehicles (HEVs). Most of the literature in this area focuses on pure simulation with limited scenarios, while real-time control through embedded implementation is still limited. In this paper, based on the basic DQN algorithm, design of mode transition control strategy, parameterized functions, and directed Wasserstein distance, the embedded scenario-adaptive D4QN-EMS to control series-parallel multi-speed (SPMS) HEVs is established. MIL and HIL tests on the strategy using standard driving cycles and nine driving cycles representing characteristics of various Chinese cities are conducted. The results show that the proposed EMS can adaptively update the strategy according to the characteristic of actual driving scenarios without frequent mode switching, demonstrating excellent performance in both transient performance and fuel economy of the SPMS HEV. Across all test scenarios, the average total fuel consumption of the A-D4QN-EMS (4.52 L/100 km) improved by 9.73%, 4.21%, and 2.43% compared to the RB-EL (4.96 L/100 km), RB-PL (4.71 L/100 km), and D4QN (4.63 L/100 km) strategies, respectively.