Efficient Energy Management of Plug-In Hybrid Electric Vehicles Through Ensemble With In-Target Minimization Q-Learning

There is a trend to implement end-to-end learning-based control for electric vehicles; however, how to ensure the optimality and safety of the learning-based control methods is a great challenge. This paper proposes a novel ensemble Q approach incorporating in-target minimization and update-to-data (UTD) mechanisms to control a plug-in hybrid vehicle that has an end-to-end control architecture. By employing these mechanisms, the prevalent issue of Q-function overestimation bias in Q-learning can be addressed with a reduction from 1.18 to 5.2 × 10<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">−4</sup>. Meanwhile, the proposed method can also improve sample efficiency, outperforming conventional Q-learning methods in terms of learning stability and performance. The advantages of the proposed method in terms of energy efficiency and learning stability were validated through experiments on software-in-the-loop (SiL) and hardware-in-the-loop (HiL) platforms. The results show that up to 10.09% and 15.59% energy savings can be achieved through the comparison with conventional Q-learning and double Q approaches.