Research on multi-stage optimization control of office building air conditioning system based on DRL

Global building energy consumption continues to increase, especially in heating ventilation air conditioning system (HVAC), so optimizing HVAC energy consumption becomes particularly important. As a key thermal system that can recover waste heat and improve energy efficiency , HP systems are widely used in HVAC. However, most of the current research focuses on local optimization of equipment inside the computer room, and in terms of optimization methods, deep reinforcement learning can effectively solve the problem that deep reinforcement learning is difficult to converge quickly. Therefore, considering the end environment, this study proposes a multi-stage model-free control method (MLC) to improve the cooling time of HP rooms in cold areas by combining DRL with traditional optimization algorithms and expert experience. Expert knowledge speeds up the agents' convergence, and the DRL optimizes the combined air conditioner's fresh air ratio (R<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">fa</inf>) and heat pump outlet temperature (T<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">hps</inf>). According to testing data, the suggested approach saves 11.6% of energy during the cooling phase as compared to HVAC under the 2022 RBC plan. This approach ensures energy savings and may be used in any type of building without relying on real data.