Model-free online reinforcement learning control for responsive building envelope

Responsive building envelopes (RBEs) represent a promising pathway toward building energy efficiency. However, their performance depends heavily on the control strategies employed. This study proposes a model-free online reinforcement learning (MFORL) control strategy using the Trust Region Policy Optimization algorithm for RBEs with variable thermal resistance. Unlike model-based methods, MFORL eliminates the need for high-fidelity modeling and significantly reduces sensing and computational costs. Co-simulation case studies across multiple climate zones and seasonal scenarios demonstrated that the MFORL controller achieved up to 93.1% energy savings and 21.4% reduction in long-term thermal discomfort. It outperformed rule-based controllers (temperature-based and demand-based) in both overall building performance and superior adaptability to environmental uncertainties. Its rapid, robust, and adaptive online learning with accelerated convergence benefits from pre-training on representative weather patterns and a low-complexity controller design. These findings highlight MFORL’s potential as a cost-efficient, adaptive, and scalable real-time control solution for dynamic building envelopes. • A cost-effective model-free online reinforcement learning control framework was developed. • Pre-training strategy significantly reduced training data size and computational costs. • Up to 93.1% energy-saving potential and 21.4% comfort improvement were demonstrated. • Low-complexity controller evolved efficiently with limited online learning experience. • Achieved superior adaptability and robustness over model-based control strategies.