Hybrid reinforcement learning optimization of aging aware energy management and powertrain sizing in fuel cell hybrid electric vehicles

This paper introduces a novel hybrid optimization framework for the aging-aware co-design of powertrain component sizing and energy management systems (EMS) in fuel cell hybrid electric vehicles (FCHEVs). The framework integrates the global search capability of the NSGA-II multi-objective evolutionary algorithm with the adaptive fine-tuning strengths of continuous deep reinforcement learning (DRL), employing Deep Deterministic Policy Gradient (DDPG), Twin Delayed DDPG (TD3), and Soft Actor-Critic (SAC). A Type-2 fuzzy logic controller is adopted as the EMS, with its parameters co-optimized alongside the scaling factors of the fuel cell stack, battery pack, and electric motor. The optimization simultaneously minimizes hydrogen consumption and component degradation, while ensuring compliance with real-world performance constraints. Validation is performed across multiple driving cycles, including the urban TEH-CAR cycle, UDDS, and WLTP-Class 3. Results indicate that, in the tested scenarios, the hybrid NSGA-II-DRL approach can improve convergence behavior and Pareto-front diversity compared to standalone NSGA-II. Fuel cell aging remained stable across algorithms, while significant differences emerged in battery degradation and fuel consumption: NSGA-II-DDPG minimized degradation but increased fuel use by 7-14%; NSGA-II-SAC reduced fuel use by 7-14% at the cost of 3-7% higher degradation; and NSGA-II-TD3 achieved fuel savings comparable to SAC with only 2-4% added degradation. Robustness tests under varying road grades further confirmed adaptability. Finally, hardware-in-the-loop validation on an STM32F7 microcontroller demonstrated real-time feasibility, with TD3- and SAC-based strategies showing superior robustness to hardware implementation effects.