Research on cooperative optimization method of source-grid-load-storage based on over-relaxation double Q-learning

The global demand for energy is increasing, along with growing awareness of environmental protection. As a new energy supply paradigm, the Energy Internet enables an efficient, clean, secure, and reliable energy supply by integrating distributed energy resources, energy storage devices, energy consumers, and energy markets into a tightly coupled system. To enhance the operational stability of multi-energy distribution networks and reduce power losses and voltage deviations, a source-grid-load-storage coordination optimization method based on over-relaxed deep double Q-learning is proposed. This approach establishes a dual-layer source-grid-load-storage collaborative optimization model, taking into account the roles and interdependencies among generation, grid, load, and storage within the coordination framework. The corresponding collaborative optimization objective functions and constraints for each layer are defined. The objective functions of the dual-layer optimization model are solved using the deep double Q-learning method, from which the Q-value of the objective function is derived. By applying the successive over-relaxation technique to optimize the Q-table, the inherent self-circulation phenomenon in deep double Q-learning is effectively mitigated, and optimal source-grid-load-storage coordination results are achieved. Experimental results demonstrate that the proposed method improves model-solving efficiency and effectively avoids self-circulation in the solution space. Compared to previous work, the comprehensive energy utilization rate exceeds 55.2%, and the voltage deviation rate at each node remains below 2%, ensuring stable operation of the multi-energy distribution network.