Adaptive reinforcement learning for energy management – A progressive approach to boost climate resilience and energy flexibility

• A novel approach for energy management based on integrating adaptive reinforcement learning, called ARLEM. • Multiple control settings are tested with different adaptive and selection features. • Three performance indicators are defined to evaluate the impact of different control approaches. • ARLEM is studied for winters in Stockholm and summers in Madrid, considering extreme climate events and multiple future climate scenarios. • ARLEM enhances climate resilience and energy flexibility in the energy network. Energy management in urban areas is challenging due to diverse energy users, dynamics environmental conditions, and the added complexity and instability of extreme weather events. We incorporate adaptive reinforcement learning (ARL) into energy management (EM) and introduce a novel approach, called ARLEM. An online, value-based, model-free ARL engine is designed that updates its policy periodically and partially by replacing less favorable actions with those better adapted to evolving environmental conditions. Multiple policy update mechanisms are assessed, varying based on the frequency and length of updates and the action selection criteria. ARLEM is tested to control the energy performance of typical urban blocks in Madrid and Stockholm considering 17 future climate scenarios for 2040–2069. Each block contains 24 buildings of different types and ages. In Madrid, ARLEM is tested for a summer with two heatwaves and in Stockholm for a winter with two cold waves. Three performance indicators are defined to evaluate the effectiveness and resilience of different control approaches during extreme weather events. ARLEM demonstrates an ability to increase climate resilience in the studied blocks by increasing energy flexibility in the network and reducing both average and peak energy demands while affecting indoor thermal comfort marginally. Since the approach does not require any information about the system dynamics, it is easy to cope with the complexities of building systems and technologies, making it an affordable technology to control large urban areas with diverse types of buildings.