PPO Based DRL for Online Solution of Optimal Configuration of GFM/GFL Generation Units in Hybrid Wind Power Station

The configuration of grid-forming (GFM)/grid-following (GFL) generation units in hybrid wind power station (HWPS) and grid strength of external power grid affect the small signal stability of the HWPS, so the time-varying and the complex operation modes of the novel power system bring greater challenges to the optimal configuration of the HWPS. First, the small signal stability of the HWPS is mainly influenced by the system short-circuit ratio of external power grid, the configuration (capacity ratio and location) of GFM/GFL generation units in the HWPS. With the goal of improving small signal stability margin and economic benefit, an optimal configuration model of GFM/GFL generation units in the HWPS is established. Secondly, since the operating modes of novel power system changes frequently causing system strength time-varying, the above optimal configuration model needs to be solved online. However, the model-driven methods or intelligent optimization algorithms suffers low efficiency and heavy computational burden, it is difficult to realize online application. In comparison, the trained deep reinforcement learning (DRL) based framework can quickly solve the optimal configuration model online. The solution problem of the optimal configuration model is transformed into reinforcement learning problem. The proximal policy optimization (PPO) algorithm introduces the dominance function and dynamic step size update mechanism, which can effectively deal with the optimal configuration model in large-scale, strong nonlinear and strong uncertainty scenarios. Therefore, the PPO based DRL method for online solution of optimal configuration of GFM/GFL generation units in the HWPS is proposed. The optimization configuration strategy generated by the proposed method can not only improve small signal stability, but also configure fewer GFM generation units to obtain better economic benefits. Finally, the trained PPO based DRL framework outputs configuration strategy online, including the capacity ratio and location of GFM/GFL generation units in the HWPS, according to the external short-circuit ratio. Taking a modified GFM/GFL HWPS with energy storage device as an example, by performance comparison with the DDPG based method under three different external short-circuit ratio scenarios, the proposed PPO based method not only reduces the number of GFM units, but also improves the small signal stability margin, which proves the superiority of the proposed method.