Developing machine learning regression model optimized using Tabu search harmony search algorithm for prediction of mass transfer in membranes

We used several advanced machine learning models for analysis of a separation unit in water purification and treatment via pressure-driven membrane. After removing outliers using the Isolation Forest method and normalizing features with a Min–Max scaler, three regression models—Gaussian Process Regression (GPR), Deep Gaussian Process Regression (DGP), and Heteroscedastic Gaussian Process Regression (HGP)—were trained and optimized for correlation of data. Hyperparameters were tuned using a hybrid Tabu Search–Harmony Search (TS–HS) algorithm. Among the models, HGP showed the strongest performance, with an R 2 of 0.9913 on the training set and 0.9846 on the test set, along with the lowest RMSE (5.91 train; 7.51 test) and MAE (3.89 train; 6.51 test). Model interpretability was assessed through SHAP analysis, confirming Time and Pressure as the most influential features. These performance values were verified using 5-fold cross-validation and supported by narrow confidence intervals, confirming the statistical reliability of the results. The results demonstrate that heteroscedastic modeling improves prediction accuracy for membrane separation system operating under variable conditions, supporting more reliable performance estimation in pressure-driven membrane applications.