Operation Optimization of Direct Renewable-to-Load System Using Deep Reinforcement Learning

Direct renewable-to-load systems have emerged as a promising pathway for improving local renewable energy utilization for large electricity consumers under constrained grid interaction and device operating limits. This paper investigates a park-level direct renewable-to-load system integrating battery energy storage and power-to-hydrogen facilities and formulates its operation problem as a sequential continuous-control task. A Deep Deterministic Policy Gradient (DDPG)-based scheduling framework is developed to explicitly model renewable generation, load demand, battery dynamics, hydrogen conversion and inventory evolution, renewable curtailment, storage-related operating cost, and the no-power-export operating boundary. Case studies based on measured wind/PV and load time-series data demonstrate that coordinated heterogeneous storage can effectively enhance system flexibility. Compared with single-type storage configurations, the coordinated battery–hydrogen scheme achieved the best overall storage performance, reducing the daily operating cost to CNY 25.28×104/day while increasing renewable energy utilization to 54.79%. Further benchmarking against MILP, GA, and a without-storage baseline shows that, although MILP remains the best offline benchmark, the proposed DDPG method provides a favorable trade-off between solution quality and online efficiency. Specifically, DDPG achieved an operating cost of CNY 20.93×104/day and a renewable energy utilization of 514.38 MWh, while requiring only 0.25 s/step for online inference. Typical-day analysis further reveals a clear functional complementarity between the two storage types; battery storage mainly provides fast short-term regulation, whereas the hydrogen subsystem mainly supports longer-duration energy shifting. These results indicate that the proposed framework offers a practical and efficient solution for the operation optimization of direct renewable-to-load systems under no-power-export constraints.