Integrating machine learning with FEM-based CFD simulation of magneto-convective nanofluid flow through corrugated quadrantal cavity with a circular obstacle

Abstract Machine learning provides a data-driven approach to predicting complex heat transfer behaviour and to reducing the computational cost of numerical simulations of magnetohydrodynamic (MHD) nanofluid systems. This study develops a machine learning framework to predict the natural convective heat transfer mechanism inside a quarter-circular enclosure containing circular obstacle filled with Cu-H₂O nanofluid under an externally applied magnetic field. The coupled governing equations are solved using the finite element method for Rayleigh number (10³-10⁶), Hartmann number (0-100), and nanoparticle volume fractions (0.0-0.05). The effects of these parameters on flow structures and temperature distribution are investigated through streamlines, isotherms and average Nusselt number analyses. The results demonstrate that increasing Rayleigh number and nanoparticle concentration enhance heat transfer rate, whereas increasing Hartmann number reduces convective motion due to magnetic damping. A dataset comprising 151 numerical simulations is used to develop the predictive models based on XGBoost, CatBoost, random forest, and artificial neural network algorithms. The models are evaluated using five-fold cross validation, where CatBoost exhibits superior predictive performance with RMSE, MAE, MSE, and R² values of 0.0488 ± 0.0342, 0.0270 ± 0.0107, 0.0033 ± 0.0046, and 0.99883 ± 0.00186, respectively. Furthermore, SHAP analysis identifies Rayleigh number as the dominant parameter for predicting the average Nusselt number and reveals the nonlinear contribution of input variables to thermal performance. The proposed machine learning framework, coupled with finite element simulations, provides a reliable and interpretable approach to predicting heat transfer characteristics in MHD nanofluid systems while reducing the computational burden of conventional numerical analysis.