Reinforcement Learning for Submodel Assignment in Adaptive Modeling of Turbulent Flames

Reinforcement learning (RL), an unsupervised machine learning approach, is innovatively introduced to turbulent combustion modeling and demonstrated through the automated construction of submodel assignment criteria within the framework of zone-adaptive combustion modeling (AdaCM). In AdaCM, the appropriate combustion submodel—whether the cost-effective species transport model or the advanced transported probability density function (TPDF) method—is adaptively assigned to different regions based on a criterion crucial for performance. The use of RL avoids the extensive manual optimization that involves repetitive calculations and struggles to account for multiple factors. Specifically, RL agents observe local variables as the state and determine the appropriate submodel through a policy. The policy is refined to maximize a reward measuring both accuracy and efficiency through the interaction between RL agents and the AdaCM solver. The methodology is demonstrated for a turbulent non-premixed jet flame, and a sophisticated RL criterion exhibiting a nonlinear and nonmonotonic dependency on the two-dimensional state of mixture fraction and Damköhler number is learned. The AdaCM with the trained criterion provides predictions that are nearly indistinguishable from those obtained using the TPDF method for the whole computational domain, while substantially reducing the computational cost with the speedup of 3.4 and only 22% of cells for TPDF.