Safe Reinforcement Learning for Power Allocation of Hybrid Energy Storage Systems in Electric Vehicles Using PPO and Predictive Safety Filter

Energy management of lithium-ion batteries (LIBs) to extend their lifespan while considering their heat generation is pivotal for their cost-effective and safe operation. For this purpose, we present a power allocation strategy for battery-supercapacitor hybrid energy storage systems (HESSs) used in electric vehicles. The proposed method combines the advantages of reinforcement learning (RL) and a predictive safety filter (PSF) to devise a safe RL solution. What is more, a tailored incentive reward is designed to guide the training processes of the RL taking into account the impacts of battery heating. Comparisons with low pass filtering (LPF), model predictive control (MPC) and twin delayed deep deterministic (TD3) policy gradient algorithms reported in the literature under various operating conditions show the superiority of the proposed approach. In particular, the proposed strategy demonstrates a remarkable reduction in the generalized operating cost, outperforming existing LPF techniques, the MPC method and the TD3 algorithm by 8.0%–30.6%, 0.3%–2.5%, and 0.3%–9.4%, respectively. Compared to MPC, the proposed method improves computational efficiency by more than double while ensuring safety. Additionally, it significantly reduces the algorithm training time by almost 97% compared to existing proximal policy optimization (PPO) methods. The proposed power allocation strategy can be applied to any system employing a hybrid battery energy storage system to alleviate battery aging while ensuring operation safety.