Generalizable deep neural networks for image quality classification of cervical images

Successful translation of artificial intelligence (AI) models into clinical practice, across clinical domains, is frequently hindered by the lack of image quality control. Diagnostic models are often trained on images with no denotation of image quality in the training data; this, in turn, can lead to misclassifications by these models when implemented in the clinical setting. In the case of cervical images, quality classification is a crucial task to ensure accurate detection of precancerous lesions or cancer; this is true for both gynecologic-oncologists' (manual) and diagnostic AI models' (automated) predictions. Factors that impact the quality of a cervical image include but are not limited to blur, poor focus, poor light, noise, obscured view of the cervix due to mucus and/or blood, improper position, and over- and/or under-exposure. Utilizing a multi-level image quality ground truth denoted by providers, we generated an image quality classifier following a multi-stage model selection process that investigated several key design choices on a multi-heterogenous "SEED" dataset of 40,534 images. We subsequently validated the best model on an external dataset ("EXT"), comprising 1,340 images captured using a different device and acquired in different geographies from "SEED". We assessed the relative impact of various axes of data heterogeneity, including device, geography, and ground-truth rater on model performance. Our best performing model achieved an area under the receiver operating characteristics curve (AUROC) of 0.92 (low quality, LQ vs. rest) and 0.93 (high quality, HQ vs. rest), and a minimal total %extreme misclassification (%EM) of 2.8% on the internal validation set. Our model also generalized well externally, achieving corresponding AUROCs of 0.83 and 0.82, and %EM of 3.9% when tested out-of-the-box on the external validation ("EXT") set. Additionally, our model was geography agnostic with no meaningful difference in performance across geographies, did not exhibit catastrophic forgetting upon retraining with new data, and mimicked the overall/average ground truth rater behavior well. Our work represents one of the first efforts at generating and externally validating an image quality classifier across multiple axes of data heterogeneity to aid in visual diagnosis of cervical precancer and cancer. We hope that this will motivate the accompaniment of adequate guardrails for AI-based pipelines to account for image quality and generalizability concerns.

Authors

Institutions

Publication Details

Journal
Scientific Reports
Published
2025-02-21
DOI
https://doi.org/10.1038/s41598-025-90024-0
Citations
4
Primary Topic
AI in cancer detection
Type
article
Field-Weighted Citation Impact
9.92

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Generalizable deep neural networks for image quality classification of cervical images

Adekunbiola A.F. Banjo, Jayashree Kalpathy-Cramer, Zeev Rosberger, Sílvia de Sanjosé et al.
4 citations
Scientific Reports
AI in cancer detection
9.92
article

Generalizable deep neural networks for image quality classification of cervical images

Adekunbiola A.F. Banjo, Jayashree Kalpathy-Cramer, Zeev Rosberger, Sílvia de Sanjosé, Karla Alfaro, Clement A. Adepiti, Kanan Desai, Teresa Norris, Karen Yeates, Nicole G. Campos, Syed Rakin Ahmed, Amna Haider, Ana Cecilia Rodríguez, José Jerónimo, Marc Steben, Muhammad Imran, Kayode Olusegun Ajenifuja, Paul Kyu Han, Juán Luis Peñaloza Figueroa, Didem Egemen, Brian Befano, Federica Inturrisi, Andrew T. Goldstein, Alex Mremi, Amélie McFadyen, Mark Schiffman, Rebecca B. Perkins, Nicolas Wentzensen, Judith Norman, Miriam Cremer, Diego Guillén, Ajenifuja Kayode Olusegun, Tainá Raiol, Safina Yuma, Ana Ribeiro, Yeycy Donastorg, Bariki Lawrence Mchome, Eyrun Floerecke Kjetland, Margaret M. Madeleine, Thay Sovannara, Te Vantha, Judy Norman, Henry B. K. Phiri, Christopher Clark, George Kassim Chilinda, Oyinloye Temitope, Bola-Oyebamiji Sekinat, Adeyemo Marydiya
article en
4 citations

Abstract

Successful translation of artificial intelligence (AI) models into clinical practice, across clinical domains, is frequently hindered by the lack of image quality control. Diagnostic models are often trained on images with no denotation of image quality in the training data; this, in turn, can lead to misclassifications by these models when implemented in the clinical setting. In the case of cervical images, quality classification is a crucial task to ensure accurate detection of precancerous lesions or cancer; this is true for both gynecologic-oncologists' (manual) and diagnostic AI models' (automated) predictions. Factors that impact the quality of a cervical image include but are not limited to blur, poor focus, poor light, noise, obscured view of the cervix due to mucus and/or blood, improper position, and over- and/or under-exposure. Utilizing a multi-level image quality ground truth denoted by providers, we generated an image quality classifier following a multi-stage model selection process that investigated several key design choices on a multi-heterogenous "SEED" dataset of 40,534 images. We subsequently validated the best model on an external dataset ("EXT"), comprising 1,340 images captured using a different device and acquired in different geographies from "SEED". We assessed the relative impact of various axes of data heterogeneity, including device, geography, and ground-truth rater on model performance. Our best performing model achieved an area under the receiver operating characteristics curve (AUROC) of 0.92 (low quality, LQ vs. rest) and 0.93 (high quality, HQ vs. rest), and a minimal total %extreme misclassification (%EM) of 2.8% on the internal validation set. Our model also generalized well externally, achieving corresponding AUROCs of 0.83 and 0.82, and %EM of 3.9% when tested out-of-the-box on the external validation ("EXT") set. Additionally, our model was geography agnostic with no meaningful difference in performance across geographies, did not exhibit catastrophic forgetting upon retraining with new data, and mimicked the overall/average ground truth rater behavior well. Our work represents one of the first efforts at generating and externally validating an image quality classifier across multiple axes of data heterogeneity to aid in visual diagnosis of cervical precancer and cancer. We hope that this will motivate the accompaniment of adequate guardrails for AI-based pipelines to account for image quality and generalizability concerns.

Scientific ReportsVol. 15(1)
Dartmouth College (US), Boston University (US), Gynecologic Oncology Group (US), Oslo University Hospital (NO), National Institutes of Health (US), Ekiti State University (NG), Cleveland Clinic (US), Harvard University (US), University of Ibadan (NG), University of Lagos (NG), University of Washington (US), Queen's University (CA), Harvard University Press (US), Adekunle Ajasin University (NG), Osun State University (NG), Massachusetts General Hospital (US), Fred Hutch Cancer Center (US), Information Management Services (US), Epicentre (South Africa) (ZA), Cleveland Research (United States) (US), Athinoula A. Martinos Center for Biomedical Imaging (US), Ministry of Health (MW), Blantyre Institute for Community Ophthalmology (MW), Kilimanjaro Christian Medical Centre (TZ), National Cancer Institute (US), Barcelona Institute for Global Health (ES), Instituto Dermatológico y Cirugía de Piel "Dr. Huberto Bogaert Díaz" (DO), Obafemi Awolowo University Teaching Hospitals Complex (NG), Center for Systems Biology (US), Obafemi Awolowo University (NG), McGill University (CA), University of Colorado Anschutz Medical Campus (US), Fundação Oswaldo Cruz (BR), Seattle University (US), Massachusetts Institute of Technology (US), Université de Montréal (CA), University of KwaZulu-Natal (ZA)
National Cancer Institute
Openalex Percentile: Top 3%
AI in cancer detection
9.92
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.