A deep reinforcement learning based EMS for PHEV considering temperature equilibrium of battery modules

Prevalent energy management strategies (EMS) for Hybrid electric vehicles (HEVs) rooted in deep reinforcement learning (DRL) encounter challenges such as suboptimal control outcomes, overlooking the consideration of inconsistent battery performance due to prolonged usage in the agent’s reward function, ultimately leading to issues like module overheating and uneven temperature distribution. This study endeavors to tackle these obstacles and introduces a suite of innovative solutions. Firstly, this study has improved the traditional Twin Delay Deep Deterministic Policy Gradients (TD3) algorithm by introducing dual Q-networks and Softmax operators to develop a new algorithm named Softmax TD3 (S-TD3), offering improved optimization of fuel economy. Furthermore, a penalty term for elevated temperatures is incorporated into the reward function. This strategic adjustment restricts the agent’s action exploration space, curtailing the frequency of battery utilization under demanding discharge scenarios, contributing to a decrease in overall battery temperature, subsequently prolonging its lifespan. Lastly, a temperature-based logic controller is devised to facilitate the secondary allocation of battery module power, which adjusts power distribution strategies in response to real-time module temperatures, promoting temperature uniformity among modules. This holistic approach not only bolsters battery thermal safety but also holds promise in further operational cost reduction.