Exclude at your own peril: evaluating the omission of data in training AI breast cancer risk models

Progress in deep learning accessibility and ease of use has led to its increased prevalence across research and clinical applications. Often, one of the first steps involves cleaning the training dataset by removing incomplete or erroneous data points. Mammograms collected from the Predicting Risk Of Cancer At Screening (PROCAS) study (n=57,897) include two views of each breast; if any case has more than four images the case is removed from the AI training set (≈5% of participants). This can occur because the breast were too large to fit in a single image, requiring a multiple ”mosaic” views, or because of a technical repeat. Other standard exclusion criteria include: presence of breast implants (≈1%), non-standard additional views (≈2%), and previous cancer diagnosis (≈3% of participants). Removal of data for these reasons can inject bias into the training dataset. If any exclusion criteria correlate to specific demographics, then this can result in a systematic misrepresentation in the AI model. This work explores the quality of deep learning models trained on ‘cleaned’ data (n=33,408) and ‘unclean’ data containing only data from study participants with more than four mammographic views (n=1,709). Models were trained to predict percentage breast density, a strong indicator of cancer risk, averaged over two expert readers’ assessments. The unclean dataset was shown to produce the same quality of model as a model trained on the 20 times larger clean dataset. This indicates that standard data cleaning practices require further assessment, given they contain enough information to produce a high-quality model. Exploring the effect of using this data can improve the amount of data available for AI training and improve the inclusion and equitable representation in technological advancements.