A Reinforcement Learning Approach for Frequency-Based Security Constrained Unit Commitment

This study investigates the capability of an offline reinforcement learning-based approach to address the dual objectives of optimal generation operation and system frequency stability. The Deep Deterministic Policy Gradient (DDPG) algorithm is used to produce generator status, initial voltage magnitude, active power generation, and reactive power generation for a given load profile and distribution. The New England 39-bus model is used as the the test case. This study generates 2,116 random scenarios and operating points with N-1 contingencies. The research findings indicate that the generator capacity utilization factor impacts frequency stability. The DDPG algorithm is able to understand the characteristics that lead to its capability in providing actions towards better stability and lower operational costs. In offline training with tabular data, DDPG achieves average rewards of 0.237, higher than random actions at 0.079.