Bayesian reinforcement learning for uncertainty-aware building energy control

The integration of renewable energy into buildings constitutes a major challenge for Home Energy Management Systems (HEMS) due to various uncertainties in demand, grid generation, and energy prices. In this paper, we formulate the HEMS control problem as a Markov Decision Process and introduce a simulation-based framework that combines Proximal Policy Optimization (PPO) with Bayesian Neural Networks (BNN) for uncertainty-aware decision-making. By using Monte Carlo dropout, our PPO-BNN approach explicitly captures epistemic uncertainty to optimize battery charging and discharging in a building simulation environment. Compared to standard PPO, our approach demonstrates superior performance, achieving a 52.1% reduction in deep-discharge events and a 14.3% improvement in annualized net economic value. These results underscore the critical role of Bayesian uncertainty quantification in developing robust, efficient, and sustainable control strategies for building energy simulation and performance optimization.