The A3C Algorithm With Eligibility Traces of Energy Management for Plug-In Hybrid Electric Vehicles

An energy management system is crucial for optimizing the performance and reducing fuel consumption of Plug-in Hybrid Electric Vehicles (PHEVs), which plays an important role in sustainable transportation. This paper presents a comprehensive study of the energy management problem for a selected PHEV, outlining key vehicle parameters such as battery state-of-charge (SOC), engine power, and energy consumption patterns. A mathematical formulation of the energy management problem is proposed to optimize fuel consumption while adhering to the vehicle’s operational constraints. The core of the paper focuses on the application of the Asynchronous Advantage Actor-Critic (A3C) algorithm to address this problem. To enhance the learning process, eligibility traces are introduced to refine the advantage function by weighting historical actions. This adjustment improves the decision-making process by making the action distribution more concentrated, which is crucial for optimizing fuel consumption. The proposed Asynchronous Advantage Actor-Critic Eligibility Traces (A3CET) algorithm incorporates these modifications and shows a significant improvement in fuel efficiency. During the training phase, it achieves 93.6% of the fuel optimality of the Dynamic Programming (DP) benchmark, and maintains 91.5% of DP’s performance during the testing phase. Moreover, A3CET reduces computation time by 62.5% compared to DP, and outperforms other EMSs in terms of both efficiency and real-time applicability.