Investigating the Impact of Imbalanced Medical Data on the Performance of Self-Supervised Learning Approaches

In clinical practice, a substantial amount of data is generated on a daily basis for diagnostic purposes.Since expensive expert knowledge is required for data annotation in order to use this data for supervised learning, large amounts of data often remain unused.Self-supervised learning methods are well suited for using unlabeled data by pre-training networks to solve pretext tasks.As medical data follow an underlying uneven distribution of occurring diseases, they are inherently imbalanced.This could introduce an unwanted bias during pre-training, ultimately leading to negative consequences that may inhibit the benefits of finetuning.In this work we investigate the impact of the imbalance of 2D and 3D medical datasets used for pre-training, as well as the importance of the type and size of the dataset used for pre-training and the pretext task.Our findings indicate that the size of the dataset used for pre-training has greater impact on the final tasks than its balance.