On the Impact of Simulated Occupancy Behavior Assumptions on Reinforcement Learning for HVAC Controls

Reinforcement Learning (RL) solutions have been extensively studied in building simulation environments for HVAC controls, providing a controlled platform for evaluating their performance.However, these simulations often rely on oversimplified assumptions, particularly with regard to human occupancy, leading to an overestimation of the controller effectiveness in real-world applications.The prevalent use of deterministic occupancy profiles fails to capture the variability of human behavior, widening the simulation-toreality gap by neglecting the very subject for which the service is designed: humans.In this work, we evaluate the impact of stochastic occupancy on RL performance by extending the state-of-the-art BEAR simulation environment.Our findings reveal significant challenges for RL algorithms in maintaining target temperatures when stochastic occupancy is introduced, calling into question the reliability of previous studies that tested controllers in the deterministic settings they were trained on.These results emphasize the need for future environments and methodologies that can robustly handle variability in occupancy assumptions.