Hierarchical Edge-Computing Framework for Grid-Responsive Building HVAC: A Privacy-Preserving Reinforcement Learning Approach

We propose a hierarchical edge-computing framework for grid-responsive building HVAC systems, which enables dynamic demand response while preserving privacy through decentralized coordination. The framework integrates microgrid-level aggregators that allocate flexibility budgets to buildings via convex optimization, lightweight reinforcement learning (RL) agents embedded in smart thermostats to optimize setpoints within these budgets, and federated learning to collaboratively refine policies without centralizing sensitive data. The aggregators minimize grid stress while respecting thermal comfort constraints, whereas the RL agents, implemented as quantized neural networks, reduce computational overhead for edge deployment. Furthermore, federated learning aggregates local policy gradients across buildings to improve global coordination, ensuring privacy through secure aggregation techniques. The proposed method interfaces with conventional HVAC systems by translating RL-based setpoint adjustments into control signals for chillers and dampers, thereby maintaining compatibility with existing infrastructure. Experimental realizations demonstrate the feasibility of the framework, with the aggregator deployed on edge servers, RL agents using TinyRL architectures, and federated learning implemented via the Flower framework. This work advances the state-of-the-art by decentralizing computational load, embedding adaptability into edge devices, and enabling privacy-aware demand response, offering a scalable solution for grid-interactive buildings