Reinforcement Learning-Based Nonlinear Optimal Discrete-Time Control of Power Systems

This paper presents a partially model-free adaptive optimal tracking control method for power systems, specifically targeting a synchronous generator connected through a reactive transmission line. By integrating the tracking error dynamics with reference trajectory dynamics, an augmented system is created. A discounted performance function is introduced to address the nonlinear tracking problem optimally. Unlike traditional methods that compute feedforward and feedback terms separately, the proposed approach calculates both simultaneously by minimizing the discounted performance function. The discrete-time tracking Bellman and Hamilton-Jacobi-Bellman (HJB) equations are derived, and a reinforcement learning (RL)-based technique is employed to solve the optimal policy online without requiring prior knowledge of system drift dynamics. Finally, the proposed method is validated through a real-time digital simulator (RTDS) with a standard power system representation.