Coordinated Short-Term Scheduling and Control Integration in Microgrids Using Dual-Layer Deep Reinforcement Learning

To optimize microgrid performance, both short-term and real-time operational objectives must be addressed, typically through energy scheduling and power control. Short-term scheduling maximizes the benefits of battery energy storage systems (BESS) by leveraging forecasted load and renewable generation conditions. However, real-time adjustments are required to account for prediction errors, as neglecting these can lead to a loss of short-term optimality and visibility in overall system performance. The challenge of coordinating scheduling with control remains underexplored due to the limited timescale difference and decoupling of optimization and control. To address this problem, a bi-level deep reinforcement learning (DRL) method is presented. The upper-level DRL optimizes short-term generation and charging/discharging schedules for synchronous generators (SGs) and BESS, respectively. The lower-level DRL implements the schedules while maintaining stability and optimality. The two DRL levels learn together in a dynamic environment to ensure the short-term and real-time operational objectives are well coordinated. Simulation results demonstrate the effectiveness of the proposed algorithm. The effectiveness of the proposed algorithm is demonstrated on the studied microgrid system. Extension to larger and more complex systems is considered as future work.