Deep-Koopman-Based Implicit Structured Q-Learning Control for Nonlinear Hydroturbine Regulation Systems

The nonlinear dynamics and varying operating conditions of hydroturbine regulation systems pose significant challenges to accurate modeling and effective control. Controllers based on simplified models or local linearizations may deteriorate beyond their calibration regions, while conventional reinforcement learning methods often suffer from high exploration complexity and unstable training. This paper proposes a Deep-Koopman-based implicit structured Q-learning strategy (DK-ISQL) for nonlinear hydroturbine regulation systems. A bilinear Deep-Koopman model is developed to represent the effects of control inputs and operating conditions through explicit state transitions in a lifted space. Based on this structure, the action-value function is formulated as a quadratic function of the control input, enabling analytical action optimization without an additional actor network. DK-ISQL was compared with PID, LQI, Koopman-MPC, DDPG, TD3, and PPO on a nonlinear physics-based hydroturbine object under varying static heads and power reference steps, with reinforcement learning convergence evaluated over five random seeds. DK-ISQL achieved average active power MAEs of 0.1674 MW and 0.1369 MW in the multi-head and reference-step tests, respectively, representing reductions of 16.8% and 26.7% compared with PID, the best-performing baseline in both tests. Across five random seeds, all DK-ISQL runs reached and remained below the 0.40 MW active power MAE threshold, converging faster and more consistently than DDPG, TD3, and PPO.