Flow separation control of an infinite wing section via multi-agent reinforcement learning

This study investigates the application of deep reinforcement learning (DRL) for active flow control in a three-dimensional NACA0012 wing section with periodic spanwise boundary conditions at low Reynolds number ( italic Re Subscript c Baseline equals 1000 <mml:math xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:mnf="http://cambridge.org/core/manifest" xmlns:cup="http://contentservices.cambridge.org" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:m="http://cambridge.org/core/metadata" xmlns:core="http://cambridge.org/core" xmlns:c="http://cambridge.org/core/content"> <mml:msub> <mml:mrow> <mml:mtext mathvariant="italic" class="MJX-tex-mathit">Re</mml:mtext> </mml:mrow> <mml:mi>c</mml:mi> </mml:msub> <mml:mo>=</mml:mo> <mml:mn>1000</mml:mn> </mml:math> $ \textit{Re}_c=1000$ ) and high angle of attack ( AoA equals 20 Superscript ring <mml:math xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:mnf="http://cambridge.org/core/manifest" xmlns:cup="http://contentservices.cambridge.org" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:m="http://cambridge.org/core/metadata" xmlns:core="http://cambridge.org/core" xmlns:c="http://cambridge.org/core/content"> <mml:mtext>AoA</mml:mtext> <mml:mo>=</mml:mo> <mml:msup> <mml:mn>20</mml:mn> <mml:mo>∘</mml:mo> </mml:msup> </mml:math> $\text{AoA}=20^\circ$ ), where the flow exhibits massive separation, strong vortex shedding and a chaotic three-dimensional wake. The baseline configuration is validated against previous numerical studies, showing the accuracy of the computational set-up. Two DRL control policies are trained with different reward functions, leading to fundamentally different control strategies. The DRL set-up uses a multi-agent reinforcement learning framework to coordinate distributed actuators along the wing span. In the first case study, when the reward prioritises drag reduction, agents delay the leading-edge separation and stabilise the wake, reducing drag by 21.1 percent sign <mml:math xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:mnf="http://cambridge.org/core/manifest" xmlns:cup="http://contentservices.cambridge.org" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:m="http://cambridge.org/core/metadata" xmlns:core="http://cambridge.org/core" xmlns:c="http://cambridge.org/core/content"> <mml:mn>21.1</mml:mn> <mml:mspace width="thinmathspace"/> <mml:mi mathvariant="normal">%</mml:mi> </mml:math> $21.1\,\%$ and lift fluctuations by 54.1 percent sign <mml:math xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:mnf="http://cambridge.org/core/manifest" xmlns:cup="http://contentservices.cambridge.org" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:m="http://cambridge.org/core/metadata" xmlns:core="http://cambridge.org/core" xmlns:c="http://cambridge.org/core/content"> <mml:mn>54.1</mml:mn> <mml:mspace width="thinmathspace"/> <mml:mi mathvariant="normal">%</mml:mi> </mml:math> $54.1\,\%$ while maintaining baseline lift. In the second case, when the reward targets lift enhancement instead, agents synchronise actuations with vortex shedding, forming leading-edge instabilities that increase lift by 42.3 percent sign <mml:math xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:mnf="http://cambridge.org/core/manifest" xmlns:cup="http://contentservices.cambridge.org" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:m="http://cambridge.org/core/metadata" xmlns:core="http://cambridge.org/core" xmlns:c="http://cambridge.org/core/content"> <mml:mn>42.3</mml:mn> <mml:mspace width="thinmathspace"/> <mml:mi mathvariant="normal">%</mml:mi> </mml:math> $42.3\,\%$ , albeit at the cost of higher fluctuations and slightly increased drag. A comparison with classical open-loop control further demonstrates that DRL autonomously determines physically meaningful and interpretable strategies consistent with established mechanisms in the literature. To the best of the authors’ knowledge, this is the first study in which a fully three-dimensional computational fluid dynamics simulation has been employed to train a DRL model for active flow control in wings, suggesting promising directions for extending DRL-based strategies to higher Reynolds numbers and more complex wing configurations where prior physical knowledge may be limited.