Time-resolved deep reinforcement learning for control of the flow past an airfoil

The current work proposes a method for the active control of flow over a National Advisory Committee of Aeronautics 0012 airfoil under turbulent condition based on time-resolved deep reinforcement learning (DRL). To leverage the coherent structures in the temporal evolution of the flow field, we integrate the long short-term memory (LSTM) network with the proximal policy optimization (PPO) method. Based on this LSTM-PPO method, the model obtained an improved strategy for controlling the mass flow rates of the three jets located on the upper surface of the airfoil to control the flow and increase the lift-to-drag ratio. The LSTM-PPO method is first compared with the traditional PPO method at Re = 2500, achieving a remarkable 160.9% enhancement of the lift-to-drag ratio. Then, the trained LSTM-PPO model is tested under several operation conditions, manifesting its adaptability. Dynamic mode decomposition is also used to study the change in the dynamics with and without the active flow control (AFC) based on the LSTM-PPO method. This study highlights the potential of recurrent neural networks to cooperate with DRL algorithms, paving the way for real-world applications of AFC.