Deep Reinforcement Learning based Model-Free Wide-Area Damping Control under Uncertain Delays

Although inter-area signals measured by PMU provides more robust control to mitigate the oscillations occurring at lower frequencies, controlling techniques relying on wide-area signals encounter considerable obstacles posed by transmission delays over longer communication channels stretching from PMUs to wide-area damping control (WADC) systems. Managing fluctuating transmission delays is crucial, particularly within expansive monitoring and control systems. Without any knowledge of the real-time transmission delays at every sample time, this paper presents pioneering method employing Deep Reinforcement Learning (DRL) for WADC, devoid of traditional modeling, to effectively mitigate inter-area low-frequency oscillations. The state set includes both cyber layer information and physical layer power system performance. The optimal control policy is designed by deep determinisitic policy gradient (DDPG) to maximize the reward function on every sample time. The reward function is designed based on the working features of the generators for enabling timely damping oscillations. The IEEE 10-Generator 39-Bus system serves as the focal point for comparative analyses conducted in this study using the proposed model-free DDPG based WADC with the proposed state set and with conventional state set. The results show that based on the proposed state set, DRL agent can get enough information of the environment. The trained optimal control policy can be continuously updated by DDPG agent with a maximum reward value. With the proposed state set, the ddpg based WADC system exemplifies the rapid suppression of inter-area low-frequency oscillations and the dependable stabilization of the power system, even when faced with uncertain time delays.