Applying Venn-Abers Predictors to Calibrate White Matter Hyperintensity Segmentations from Brain Images

Motivation: Uncertainty is critical in making informed decisions from the results of machine learning models, however accurately assessing uncertainty relies on the calibration of the underlying model. Goal(s): To improve the performance and calibration of a white matter hyperintensity segmentation tool. Approach: Inductive Venn-Abers predictors were used which guarantees good model calibration performance subject to reasonable assumptions. We leveraged an open-source dataset for calibration and testing. Results: Substantial improvement in calibration metrics were demonstrated, with log-loss being halved and near-perfect calibration being obtained when the assumption of exchangeability was met. Furthermore, the calibrated method demonstrated improved post-threshold performance and a reduction in volumetric bias. Impact: We demonstrated that Inductive Venn-Abers Predictors can be used to reliably calibrate a deep-learning segmentation tool, which improved model performance, calibration, uncertainty estimates, and aids in the interpretability of the resulting segmentation maps