Comparing Deep Reinforcement Learning Methods for HVAC System Optimization with Forecast Parameters Over Various Time Periods

Buildings are significant contributors to global energy consumption. Maintaining comfortable indoor temperatures while reducing energy consumption are conflicting objectives. Deep Reinforcement Learning (DRL) is a promising area of research for building Heating, Ventilation and Air Conditioning (HVAC) system optimization. In this study an open-source framework Building Optimization Testing Framework (BOPTEST), which is a virtual testbed that help comparison different control strategies for evaluation of DRL control methods is used. A Proportional-Integral (PI) controller is used to benchmark the DRL methods. A single zone residential building of 192 m<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> with a radial heating system and a heat pump in a climate zone with high heating requirement with dynamic electricity prices with prices varying every 15 min based on demand is chosen for implementing different control strategies. On comparing Deep Deterministic Policy Gradient (DDPG), Proximal Policy Optimization (PPO) and Twin Delay DDPG (TD3) based DRL controllers and the baseline controller, the DDPG based controller reduced energy consumption by 97.3 % and operating cost by 17.7 % during the peak heating period with reference to baseline method. Then on analyzing the impact of inclusion of forecast parameters occupancy, solar irradiance, and electricity prices over the period 3, 6 and 12 hours in DDPG based controller. The prediction for 3 hours gave the greatest reduction in thermal discomfort of 99.7 % and prediction for 12 hours gave maximum reduction in cost by 30.4 % but resulted in only 82% reduction in thermal comfort when compared with baseline method indicating that longer prediction horizon is not necessarily results in better performance.