Deep reinforcement learning-based coordinated optimization between distribution networks and microgrids towards demand response uncertainty

• A hierarchical interaction framework is proposed for distribution networks. • Deep reinforcement learning is used to optimize dispatch scheme for microgrids. • Multiple armed bandit algorithm is adopted to learn user behavior uncertainties. The transition of modern power systems towards high renewable penetration demands advanced artificial intelligence solutions to mitigate the inherent intermittency and uncertainty of distributed energy resources. Traditional optimization methods often struggle to effectively handle demand response uncertainty, thereby posing risks to grid security. To address this challenge, this paper proposes a hierarchical coordinated optimization framework integrating distribution networks, microgrids, and users. Specifically, the interaction between distribution networks and microgrids employs a multi-agent reinforcement learning architecture, where an actor-critic network is utilized to generate optimal power dispatch commands for the microgrids, adaptively accommodating varying response capabilities. The interaction between each microgrid and its internal users is modeled as a multi-armed bandit problem, incorporating an online learning algorithm integrated with the upper confidence bound strategy is applied to dynamically optimize user combinations, consequently enhancing the reliability and guaranteed performance of user response. Finally, the effectiveness of the proposed method is validated through simulations on a modified IEEE 33 node system. The results demonstrate that the proposed method reduces the root mean square error of power response by 50.58% and decreases the mean absolute percentage error by 49.84%, effectively mitigating the impact of demand response uncertainty. Furthermore, the total operating cost is reduced by 8.06%, verifying the economic superiority of the proposed framework.