QLSTM-Based Microgrid Daily Operation with Renewable Uncertainty

Microgrid is a small-scale grid where generation is close to the demand allowing more penetration of renewables, like photovoltaic (PV). However, the intermittent nature of PV power generation poses a significant challenge in microgrid operation, especially on days with highly variable weather conditions. In this paper, a deep reinforcement Q-learning long short-term memory (QLSTM) model is developed to predict the operation strategy of a microgrid for the next day at a 15-minute time interval. To address the uncertainty of PV power and demand, the previous three days' PV and load data are added as inputs to the model since weather conditions on consecutive days may depend on similar atmospheric conditions. Also, to address the effect of propagation of error in the long forecasting horizon with multiple steps, a moving window training method is implemented. The moving window will be shifted by 15 minutes at each step once the actual PV and load data are available till the end of the day. The model is tested in a microgrid consisting of combined cooling, heating and power, heat pump, PV, battery, and heating and cooling energy storage systems. Results show that our model outperforms gated recurrent unit, LSTM, and Q-learning for testing data from different months. Also, it shows better performance than MATLAB 2023 Optimization Toolbox (the branch-and-bound method) which uses forecasted data, especially on a day with highly variable weather conditions.