Deep reinforcement learning based energy management for hybrid electric buses under uncertain operational states

With the rapid development of automotive intelligence, energy management strategies (EMS) for hybrid electric buses (HEBs) face higher demands. Traditional EMS cannot effectively handle dynamic changes in operational states, such as time-varying vehicle mass and road slope. To address this, this study develops a hierarchical EMS that accounts for uncertain operational states and proposes a novel EMS optimization algorithm based on deep reinforcement learning (DRL) for torque distribution in HEBs. First, to improve the adaptability of the EMS to various regions and road conditions, Extended Kalman Filtering (EKF) is used to jointly estimate real-time vehicle mass and road slope. Combined with historical operating data, Dynamic Programming (DP) performs global planning of operating conditions. The optimal experiences are integrated into experience replay and considered in reward function. Second, to tackle the sparse reward issue during training, the Twin Delayed Deep Deterministic Policy Gradient (TD3) framework is enhanced with annealing bias-priority experience replay and sample learning rate annealing, improving training efficiency. Finally, the proposed strategy is validated in a synthesized driving environment for fixed-route bus operations. Validation results show that, compared to traditional TD3-based EMS, the proposed method improves learning capability by 51.0%, convergence speed by 13.1%, and fuel economy by 9.4%.