Decomposed-Critic Multi-Agent Deep Reinforcement Learning for Multi-Zone HVAC Control

Deep Reinforcement Learning (DRL) has proven to be highly effective in improving the energy efficiency of Heating, Ventilation, and Air Conditioning (HVAC) systems. However, in multi-zone HVAC systems, DRL faces challenges in balancing the entangled rewards of zone-level thermal comfort and total HVAC energy consumption, which can potentially lead to suboptimal policies. To address this issue, Decomposed-Critic Multi-Agent Deep Reinforcement Learning (DCMADRL) is proposed. In this algorithm, the critic is split into two components: a global critic that evaluates the state-action value with respect to total HVAC energy consumption, and a local critic that evaluates the state-action value in relation to thermal comfort in individual zone. The algorithm was validated using a multi-zone HVAC model in EnergyPlus. Experimental results indicate that DCMADRL achieves energy savings of up to 12.95% compared to baseline methods, while effectively maintaining thermal comfort.