Demand response optimization of gas-electric water heater based on fusing knowledge and reinforcement learning

Gas-electric water heater (GEWH) is an important load type for integrated demand response (IDR), in which the IDR optimization strategy is required of fast self-adaptiveness to conquer uncertainties in the load itself and in operation environment of GEWH. This paper investigates a solution that integrates knowledge in deep reinforcement learning (DRL) method. We establish an optimization structure that couples the physical device, device model and optimization strategy automatically. We set up rule-based knowledge for IDR optimization. Furthermore, we design a DQN (deep Q-learning)-based optimization model with knowledge integration including common features of DQN, the method that optimization knowledge works in reward function and the control mechanism that coordinates the depthand probability of knowledge participation. Our case studies show that the proposed method is able to automatically adapt to the uncertainties in GEWH load and its working environment and converge to the optimal solution. Compared with the demand response of electric water heater, IDR for GEWH reduces the energy cost by 18.7%. Moreover, the proposed method outperforms standard DQN by five times the convergence rate, which provides references for large-scale IDR optimization implementation for GEWH.