A Resilient Reinforcement Learning based Home Energy Management System

Home energy management systems (HEMSs) automate the control of energy consumption in buildings, optimizing energy savings and cost efficiency while maintaining comfort constraints.Data-driven methods, particularly reinforcement learning (RL), are gaining increasing popularity as control strategies in such systems.However, existing research often assumes error-free, ideal datasets, which are rare in real-world applications.Sensor inaccuracies, hardware failures, or disrupted data flows due to unstable network connections can significantly impair system performance.This paper presents a novel approach to RL based HEMS controllers by integrating a Graph Neural Network (GNN) with a Transformer Soft Actor-Critic (TSAC) algorithm, aiming to improve system resilience.The proposed framework integrates GNN layers into the TSAC model's state estimator and policy function, leveraging spatial information for improved handling of incomplete and noisy data.Performance is assessed on a real-world dataset augmented with artificial perturbations.A standalone TSAC model is used as the baseline algorithm, while an MPC method with perfect predictions serves as the upper benchmark for comparison.The results reveal that the approach matches the performance of the MPC method while significantly enhancing resilience, ensuring critical temperature and comfort constraints are maintained under challenging conditions.Moreover, training the TSAC model directly on perturbed data further improves its robustness, achieving resilience comparable to the combined GNN-TSAC framework but with fewer parameters and reduced computational cost.These findings highlight that training deep neural network architectures capable of processing historical observations on perturbed datasets via RL not only increases resilience but also enhances overall performance.