Optical Water Type Guided Benchmarking of Machine Learning Generalization for Secchi Disk Depth Retrieval

Secchi disk depth (SDD) is a widely critical indicator of water transparency. However, existing retrieval models often suffer from limited transferability and biased predictions when applied to optically diverse waters. Here, we compiled a dataset of 6218 paired in situ SDD and remote sensing reflectance (Rrs) measurements to evaluate model generalization. We benchmarked nine machine learning (ML) models (RF, KNN, SVM, XGB, LGBM, CAT, RealMLP, BNN-MCD, and MDN) under three validation scenarios with progressively decreasing training-test overlap: Random, Waterbody, and Cross-Optical Water Type (Cross-OWT). Furthermore, SHAP analysis was employed to interpret feature contributions and relate model behaviors to optical properties. Results revealed a distinct scenario-dependent generalization gradient. Random splits yielded minimal bias. In contrast, Waterbody transfer consistently shifted predictions toward underestimation (SSPB: −16.9% to −3.8%). Notably, Cross-OWT extrapolation caused significant error inflation and a bias reversal toward overestimation (SSPB: 10.7% to 88.6%). Among all models, the Mixture Density Network (MDN) demonstrated superior robustness with the lowest overestimation (SSPB = 10.7%) under the Cross-OWT scenario. SHAP interpretation indicated that engineered indices, particularly NSMI, functioned as regime separators, with substantial shifts in feature attribution occurring at NSMI values between 0.4 and 0.6. Accordingly, feature sensitivity analysis showed that removing band ratios and indices improved Cross-OWT robustness for several classical ML models. For instance, KNN exhibited a significant reduction in Median Symmetric Accuracy (MdSA) from 96% to 40% after feature reduction. These findings highlight that model applicability must be evaluated under scenario-specific conditions, and feature engineering strategies require rigorous testing against optical regime shifts to ensure generalization.