Between-study variance inflates apparent accuracy in literature-compiled machine learning models of sulphate resistance in SCM-blended cement

Abstract Machine learning models trained on compressive strength data aggregated from published experimental studies are increasingly used to predict the sulphate resistance of cement blended with supplementary cementitious materials (SCMs), routinely reporting coefficients of determination above 0.90. This study examines whether that accuracy reflects genuine predictive capability. A dataset of 94 observations was compiled from three peer-reviewed studies covering silica fume, nano-silica, fly ash, slag, rice husk ash and cassava peel ash at replacement levels of 0–30%, exposure durations of 0–500 days and sulphate concentrations of 0.352–0.704 mol/L. A ridge regression model (α = 0.078) achieved R² = 0.968 on a randomly held-out test set, comparable to ordinary least squares (0.968), Lasso (0.966) and random forest (0.982). Under leave-one-study-out validation, however, every fold returned a negative R² (− 12.28 to − 162.35), with mean absolute errors up to 42.5 MPa on mortars whose measured strengths did not exceed 21.5 MPa. Variance decomposition showed that 96.1% of total strength variance lay between source studies rather than within them. Ten of eleven SCM categories occurred in only one study, and each study used its own sulphate cation and concentration, rendering material effects and study effects mathematically inseparable. For the single SCM type present in two studies, mean strengths differed by 7.3 MPa across studies, approximately half the 14.0 MPa range observed across all nine SCM types within a single study. Random train–test splitting cannot detect this confounding. Leave-one-study-out validation and between-study variance reporting should be treated as minimum requirements for literature-compiled durability models.