A Preference-Based Online Reinforcement Learning With Embedded Communication Failure Solutions in Smart Grid

Microgrids, marked by substantial renewable energy integration, have garnered significant attention. Traditional optimization methods face challenges in handling the unpredictability of renewable energy, market prices, and loads. Reinforcement learning (RL) offers a solution by learning from historical data. However, Offline-RL models often encounter adaptability challenges in new environments, while traditional Online-RL faces stability issues in certain scenarios. To address these concerns, this article proposes a preference-based deep deterministic policy gradient (PDDPG) algorithm. It guides the online learning process using expert experience based on expert rules and offline-RL to enhance the model's performance. Moreover, unlike studies that assume perfect communication and ignore random communication failures, the proposed real-time microgrid energy management system tackles communication challenges by incorporating a communication detection and data supplement system (CDDSS), especially during extreme weather conditions. The results indicate that the incorporation of CDDSS, as opposed to zero value supplementation, previous moment data, and conventional predictive models, results in a remarkable reduction of microgrid losses by 88.3%, 80.5%, and 53.4%, respectively.