Reviving Poor Object Segmentations in OOD Medical Images using Variational-Deep-PCA Modeling on Segmentation Maps with Sampling-Free Learning

For object segmentation in medical images, deep neural networks (DNNs) typically perform poorly on out-of-distribution (OOD) images stemming from the large variability in image-acquisition equipment and protocols across sites. However, we observe that the variability in the underlying object-segmentation maps is far lower. Thus, we propose a novel DNN framework to model this variability in segmentation maps, and leverage it to revive poor segmentations produced by existing DNNs on OOD images. Our DNN framework (i) learns the principal modes of variation in a class of segmentation maps, (ii) models each segmentation map using a low-dimensional mixture-of-modes latent representation on a simplex, (iii) enables sampling-free variational learning and uncertainty estimation, and (iv) trains using small in-distribution image sets. In special cases when OOD-image segmentations are extremely poor, we propose a human-in-the-loop method needing minuscule human intervention. Results using 6 publicly-available datasets and 8 existing DNN segmenters show the benefits of our framework in OOD-image object segmentation.