Uncertainty Estimation in Predicting Suspended Sediment Load Using Probabilistic Machine Learning and Conformal Prediction

This study reports the results of integrating conformal prediction based uncertainty estimation approaches with four machine learning models, i.e., light gradient boosting machine (LightGBM), extreme gradient boosting (XGBoost), natural gradient boosting (NGBoost), and probabilistic gradient boosting machine (PGBM), for predicting river suspended sediment load. Conformal prediction provides uncertainty estimates by wrapping around a predictive model to output intervals guaranteed to contain the true value with a predefined confidence level. It works by dividing training samples into training and validation sets, for which several approaches are proposed. This study specifically evaluates three data splitting approaches: split conformal prediction, cross validation plus (CV+), and conformal quantile regression. Since LightGBM and XGBoost inherently provide only point estimates, an instance-based approach was used to modify these algorithms for probabilistic prediction. The resulting predictions were then compared with those obtained from NGBoost and PGBM. To evaluate the potential of all algorithms for point prediction, probabilistic prediction, and uncertainty estimation, a dataset consisting of 1,460 field observations for discharge and suspended sediment load was used. Of the total dataset, 1,095 samples were used for training, and the remaining 365 samples were used for testing. The AutoSampler package within Optuna, a Python-based hyperparameter optimization framework, was used to determine optimal values of user-defined parameters for all machine learning models. The quality of probabilistic prediction was assessed using parameters including the sharpness, continuous ranked probability score (CRPS), negative log likelihood, probability integral transform histogram, calibration curve, and CRPS decomposition. For conformal prediction, the mean predicted interval width, effective coverage, and coverage width criterion were used. Results indicate that PGBM achieved the best performance for point prediction with a correlation coefficient of 0.972, mean absolute error of 89.790 t/D, root mean square error of 289.633 t/D, and a combined accuracy of 125.228 t/D. For probabilistic prediction, results in terms of various indicators suggest that NGBoost is the best-performing model. Regarding uncertainty estimation, the PGBM model coupled with the CV+ based adaptive CP approach achieved the highest coverage, effectively capturing even heavy-tailed data.