Uncertainty-aware Masked Modeling in Medical Imaging

Denoising the corrupted images using an autoencoder is a promising approach to obtain a good performing encoder. This sub-field, known as MIM (Masked Image Modeling), has gained renewed attention with the advent of Masked Autoencoders (MAE). Although MAE and its subsequent work have revealed the significant impact of masking strategies on downstream task performance, existing masking strategy designs remain relatively basic. In this paper, we propose UAM<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup>, a novel uncertainty-aware masked modeling framework for universal encoder pre-training on medical CT images. Our framework effectively leverages the model’s perceptual capacity during training by encouraging uncertainty prediction for different patches under controlled perturbations. Specifically, the framework adopts the mean teacher paradigm, consisting of a teacher model and a student model with the same architecture. During training, the teacher model generates challenging masks, while the student model completes the preset reconstruction task and updates the teacher model using exponential moving average (EMA). The core is an uncertainty-aware scheme that allows the student model to use predictive uncertainty information to progressively focus on high information entropy contents. Comprehensive experiments demonstrate that UAM<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> outperforms state-of-the-art MIM methods, highlighting the potential of our framework for addressing challenging medical downstream problems.