Real-Time MDP-Based Optimal Control Strategy for Maximizing Green Hydrogen Production in PV Microgrids

Off-grid photovoltaic (PV) systems require resilient energy storage to overcome power intermittency. While conventional batteries provide short-term buffering, they struggle to maintain long-term energy management. Green hydrogen emerges as a complementary solution, serving both as a scalable long-term storage medium and a clean energy carrier. However, maximizing hydrogen generation remains challenging due to the variability of solar irradiance and load demand. This paper proposes a stochastic optimal control strategy for autonomous PV systems combining battery storage with a proton exchange membrane (PEM) electrolyzer. The energy management problem is formulated as a Markov decision process (MDP), enabling real-time decisions under uncertain operating conditions. The controller prioritizes load supply, regulates battery state of charge (SoC), and allocates surplus PV energy to hydrogen production. Simulation results demonstrate that the proposed MDP-based optimal controller significantly outperforms the conventional perturb-and-observe (P&O) method. In particular, it has a reduced average tracking error of 0.344 instead of 7.517, an improved energy conversion efficiency of 99.9% compared to 96.5%, and converges to the optimal operating point within 0.1s instead of 0.5s. These improvements enhance energy utilization, stabilize hydrogen output, and extend battery lifespan. Overall, the findings confirm the effectiveness of stochastic control for optimizing hydrogen production in standalone PV microgrids, reinforcing their role in advancing the clean energy transition.