A Q-learning based Adaptive Control of Hybrid Energy Storage System to Mitigate Power Fluctuations in Grid-Connected Microgrids

The intermittent and fluctuating output of wind turbines is increasingly recognized as a major issue affecting the power quality and stability of electrical grids. As wind power integration grows, addressing this challenge is essential. A promising solution involves using a Hybrid Energy Storage System (HESS), combining battery energy storage systems (BESS) and supercapacitors (SC). This paper presents a Q-learning based control strategy for tuning the parameters of a two-stage variable time constant low-pass filter (LPF) in a grid-connected microgrid. The proposed strategy adaptively adjusts the LPF time constant to mitigate wind power fluctuations. It also accounts for practical constraints of energy storage systems and their interfaced converters, such as preventing overcharge/discharge and adhering to maximum power conversion limits. Numerical simulations confirm the effectiveness of the two-stage variable time constant LPF in reducing output wind power fluctuations while considering the practical constraints of HESS.