Deep Reinforcement Learning for Price-Aware Building Heating Control

Abstract Heating systems account for a significant share of residential energy consumption, and rising energy prices call for intelligent, cost-aware control strategies. Traditional methods, such as rule-based or model predictive control (MPC), often require detailed system modeling or lack adaptability to dynamic price signals. This work explores the use of deep reinforcement learning (DRL) to control heat pumps in a way that balances occupant comfort with energy-cost minimization. We evaluate deep Q-network (DQN) and proximal policy optimization (PPO) methods across discrete and continuous action spaces. The agents are trained in simulation using real weather and electricity price data, with a model representing the thermal dynamics of the building. Short-term electricity price forecasts are included to enable anticipatory heating strategies. Reward functions combine price penalties with piecewise-linear or quadratic comfort penalties. Among the DRL variants, a DQN agent with discrete actions and a piecewise-linear comfort reward achieves the best overall trade-off between comfort and cost. MPC still performs best in absolute cost terms because it uses an exact model, while the DQN policy approaches MPC performance and retains the model-free, adaptive advantages of RL. The findings highlight the potential of DRL for adaptive and price-aware heating control without the need for detailed physical modeling.