Double-diffusive natural convection of ternary nanofluids in a trapezoidal enclosure using neural computing and sensitivity analysis

Purpose The purpose of this study is to integrate three approaches, finite element simulation, response surface sensitivity analysis and Levenberg–Marquardt scheme-artificial neural network (LMS-ANN) prediction, to accurately evaluate average heat and mass transfer rates in a trapezoidal cavity filled with ternary nanofluids under double-diffusive convection. By combining numerical modeling, statistical optimization and machine-learning forecasting, the study aims to deliver a comprehensive and reliable framework for enhancing thermal-mass transport performance in thermal engineering applications. Design/methodology/approach The governing equations are numerically solved using the finite element method. A sensitivity analysis is performed through response surface methodology based on a central composite design and analysis of variance to quantify the influence of the key parameters. Furthermore, a LMS-ANN, optimized using gradient descent and using a tangent-sigmoid activation function, is developed in MATLAB to accurately predict the average Nusselt and Sherwood numbers. Findings Results show that an increase in the nanoparticle volume fraction coefficient leads to a higher Nusselt number, while the local Sherwood number decreases. An increase in the Lewis number reduces fluid circulation and raises the temperature and concentration within the cavity. Regression plots reveal that the coefficient of determination reaches unity for both the average Nusselt and Sherwood numbers. Sensitivity analysis identifies the Rayleigh number as the most dominant parameter influencing the average Nusselt and Sherwood numbers. Originality/value This study presents an integrated computational, statistical and machine learning framework to analyze ternary nanofluid behavior in a trapezoidal cavity, providing a comprehensive approach for evaluating coupled heat and mass transfer in energy-efficient thermal systems.