Dual-Critic Multi-Agent Deep Reinforcement Learning for Multi-Zone HVAC Safety Control

Deep Reinforcement Learning (DRL) provides a novel solution for optimizing energy efficiency in Heating, Ventilation, and Air Conditioning (HVAC) systems. However, in multizone HVAC systems, DRL faces challenges in balancing temperature constraints with energy efficiency, making it difficult to effectively ensure safety control of HVAC, maintaining the indoor temperature within the safe range. To address this issue, a Safe Dual-Critic Multi-Agent Deep Reinforcement Learning (SDC-MADRL) algorithm is proposed. The algorithm decomposes the critic network into two components: a reward network and a cost network, which correspondingly evaluate energy consumption and the violation of temperature constraints. Furthermore, a Lagrange multiplier mechanism is introduced to dynamically adjust the temperature constraint weights, ensuring that the temperature consistently meets the control requirements. The algorithm is validated using a multi-zone HVAC model in EnergyPlus. Experimental results show that, compared to baseline methods, SDC-MADRL significantly reduces the probability of temperature violations (down to 0.626%) while effectively reducing system energy consumption.