Uncertainty-Informed Active Learning Using Monte Carlo Dropout for Risk Stratification in Carotid Ultrasound Imaging

The integration of model uncertainty quantification in clinical decision support systems, incorporating machine learning models, can augment the models’ reliability and robustness against domain shifts while also promoting user confidence and trust. In the present study, an uncertainty-informed active learning approach, leveraging Monte Carlo dropout for uncertainty estimation, is proposed towards the development of a deep learning model able to classify carotid ultrasound images as high-risk and low-risk for cardiovascular disease. An auxiliary dataset (CUBS) is employed for the initial model's development and fine-tuning as well as the optimization of the Monte Carlo dropout's hyperparameters. A dataset (87 B-mode ultrasound sequences) from ATTIKON hospital is subsequently utilized within the framework of active learning for model retraining based on the selection of the most informative samples according to the Monte Carlo dropout uncertainty estimation. In this context, the use of three active learning strategies is investigated, including uncertainty rank selection, pseudo-labeling for certain samples, and pseudo-labeling with variable sample weighting. The obtained results indicate that pseudo-labeling with variable sample weighting yields the best performance, achieving an AUC of 87.28% with only 21 annotated samples, which account for 30% of the total training data. Thus, this work provides evidence regarding the ability of uncertainty quantification and active learning to reduce labeling costs while maintaining model performance and enhancing the robustness and reliability of cardiovascular risk prediction models.