Improving Demand Response Programs Using Override Signals with Reinforcement Learning

Demand-side management is becoming essential for achieving global energy objectives and enhance the integration of intermittent renewable generation.However, in residential demand response (DR) programs, signal overrides by consumers, often driven by thermal comfort concerns, can undermine the targeted objectives.Modeling override behavior is challenging due to limited access to behind-the-meter data.This study proposes an offline-to-online reinforcement learning (RL) approach to improve pre-existing DR policies using consumer feedback, such as direct override signals, along with only weather and smart-meter data.Using the Calibrated Q-Learning (CalQL) algorithm, we demonstrate the potential to enhance DR outcomes in a simulated air-conditioner-based system involving 100 residential consumers.The results show significant improvements in peak load reduction (up to 60%) and override minimization (up to 99%) while maintaining scalability and practicality for real-world deployment.We study two distinct initial policy types, and further analyze the main parameters influencing such learning paradigms to discuss their impact on DR outcomes.