Optimal energy management in smart energy systems: A deep reinforcement learning approach and a digital twin case-study

This research work introduces a novel approach to energy management in Smart Energy Systems (SES) using Deep Reinforcement Learning (DRL) to optimize the management of flexible energy systems in SES, including heating, cooling and electricity storage systems along with District Heating and Cooling Systems (DHCS). The proposed approach is applied on Meridia Smart Energy (MSE), a french demonstration project for SES. The proposed DRL framework, based on actor–critic architecture, is first applied on a Modelica digital twin that we developed for the MSE SES, and is benchmarked against a rule-based approach. The DRL agent learnt an effective strategy for managing thermal and electrical storage systems, resulting in optimized energy costs within the SES. Notably, the acquired strategy achieved annual cost reduction of at least 5% compared to the rule-based benchmark strategy. Moreover, the near-real time decision-making capabilities of the trained DRL agent provides a significant advantage over traditional optimization methods that require time-consuming re-computation at each decision point. By training the DRL agent on a digital twin of the real-world MSE project, rather than hypothetical simulation models, this study lays the foundation for a pioneering application of DRL in the real-world MSE SES, showcasing its potential for practical implementation. • Deep Reinforcement Learning for optimal energy management in Smart Energy Systems. • A DDPG algorithm for multiple scheduling actions on multi-energy storage systems. • Reward shaping to address sparse reward issues faced by DRL agents. • Application on a digital twin of the Meridia Smart Energy (MSE) real-world SES.