Price-responsive control using deep reinforcement learning for heating systems: Simulation and living lab experiment

While buildings are one of the main sources of global carbon emissions, they also present significant opportunities for decarbonization. Building energy systems are typically controlled using pre-defined schedules and rule-based strategies, which usually lack adaptability to dynamic conditions such as changing weather states, occupancy and energy prices. In this study, we develop a price-responsive Deep Reinforcement Learning (DRL) controller to autonomously control the indoor temperature of a building. The designed controller aims to minimize heating costs while keeping the indoor temperature within the acceptable comfort bounds, responding to dynamic energy prices. To improve learning efficiency, this study introduces Temporally-Weighted State Difference (TSD), a method to compress future price data into a single state observation for the controller, enabling future-aware decisions while reducing training complexity. The controller is developed and trained in a simulation environment using a physically consistent neural network model. It is then implemented in a real-world residential unit in the NEST building in Dübendorf, Switzerland, and evaluated during the winter of 2024 to 2025 using the dynamic electricity prices of the Swiss market. Experimental results indicate that incorporating future price information leads to more informed decisions by the controller and improves the performance of DRL by 11.5%. Experimental results show a 79% reduction in heating costs compared to a rule-based controller, at a cost of exceeding the thermal comfort range by 0.3 °C on average. The study demonstrates the potential of price-responsive DRL in reducing energy costs and providing demand-side flexibility to the energy grid. • Temporally-Weighted State Difference is introduced to improve the efficiency of DRL. • DRL could reduce heat costs by 79% compared to rule-based control. • Price-responsive DRL could get 11.5% higher cost saving compared to ordinary DRL. • Price-responsive DRL can provide flexibility to the grid while reducing heating bills.