Reinforcement-Learning-Based Hierarchical Resilience-Oriented Microgrid Energy Management

Microgrid energy management must reconcile economic operation with operational resilience under supply-loss contingencies and power fluctuations. Economically efficient schedules may leave insufficient dynamic capability to contain severe frequency excursions following emergency islanding or generator trips. This paper proposes a reinforcement-learning-based hierarchical energy-management framework that couples resilience-oriented day-ahead scheduling with forecast-informed real-time dynamic support. The day-ahead layer jointly optimizes generation, battery energy storage, utility exchange, and baseline dynamic-support capability subject to post-contingency frequency-security constraints. Building upon this baseline, the real-time layer adaptively coordinates the virtual inertia and damping of grid-forming battery converters using local measurements, short-horizon load forecasts, and available device capability. Soft actor–critic (SAC) serves as the common continuous-action learning algorithm across both hierarchical layers. Case studies show that the proposed framework maintains the prescribed frequency-security requirements across the evaluated supply-loss contingencies, battery-health conditions, and fast and sustained load variations. The day-ahead SAC costs differ from the optimization reference by less than 0.4% in both operating modes, while the maximum measured software-side supervisory computation remains below 0.3 s over the tested real-time cases. These results demonstrate that dynamic frequency-support capability can be preventively allocated through scheduling and adaptively deployed as operating conditions evolve.