Reinforcement Learning Environment for Demand Controlled Cabin Ventilation in Demand Response

This paper explores strategies for controlling demand-controlled ventilation in a cruise ship heating, ventilation, and air conditioning (HVAC) system during demand-side management (DMS) operation. The HVAC system is the second largest energy consumer on a cruise ship after propulsion and therefore suitable for DSM operation. The energy load of HVAC system can be described as distributed. It can be assumed that the best trade-off between air quality violations and energy savings can be achieved by aggregating energy savings from multiple cabins. The energy savings from an HVAC system is a complex challenge as automation in different parts of the HVAC system tries to compensate for air quality violations. Thus, energy saving needs to be done holistically. The demand controlled ventilation (DCV) system allows better controllability with the cost of damper-related problems such as drifting in measurements. In normal operations such problems can be handled with reconfiguration, but during the DSM, especially time-critical demand response (DR) operations, it can cause unwanted outputs for energy saving or air quality. Reinforcement learning (RL) has shown potential in decision making under uncertain conditions. This work-in-progress paper suggests an implementation of RL environment under DR policy to optimize the usage of DCV system, while taking into account the reconfiguration situations.