OC01.04: *AI‐driven ultrasound detection of ovarian cancer that generalises: an international multicentre validation study

A critical shortage of expert ultrasound examiners has raised concerns of unnecessary interventions and delayed diagnoses. Artificial intelligence (AI)-driven diagnostic support has the potential to alleviate this burden and improve patient outcomes. Deep learning has shown promising results in ovarian cancer detection; however, external validation is lacking. Our aim was to develop a deep learning model for ovarian cancer detection and evaluate its robustness and ability to generalise in a large multicentre setting. We retrospectively collected 17,119 ultrasound images from 3,652 women with an ovarian lesion (2,224 benign, 1,428 malignant) from 20 centres in eight countries. A total of 2,718 cases were reviewed by a minimum of 7 expert and 6 non-expert examiners. For each centre in turn, we trained a transformer-based model using data from the remaining centres and compared the models' performance with that of human examiners. Furthermore, we simulated and assessed how these models could be used in AI-assisted triage and compared it to current practice. The models demonstrated robust performance across centres, ultrasound systems, and histological diagnoses, with an area under the receiver operating characteristic curve of 0.93 (95% CI 0.92–0.94) and F1 score of 83.7% (95% CI 82.3–85.0) on cases from unseen centres. They outperformed both expert and non-expert examiners, with F1 scores of 79.6% (95% CI 78.3–81.3) and 74.7% (95% CI 73.0–76.3), respectively. The models were further shown to produce well-calibrated predictions. In a retrospective simulation, AI-assisted diagnostic support reduced the number of referrals to experts by 63% compared to current practice, while increasing the diagnostic performance (F1 83.0% vs 77.5%). The models exhibit strong generalization and outperform expert examiners. Introducing AI-driven diagnostic support into the clinical workflow may reduce human resource demands, while improving diagnostic performance.

Authors

Institutions

Publication Details

Journal
Ultrasound in Obstetrics and Gynecology
Published
2024-09-01
DOI
https://doi.org/10.1002/uog.27724
Citations
1
Primary Topic
Ultrasound and Hyperthermia Applications
Type
article
Field-Weighted Citation Impact
0.18

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

OC01.04: *AI‐driven ultrasound detection of ovarian cancer that generalises: an international multicentre validation study

Paula Herman, Ekaterini Domali, S. Guerriero, M. Pascual et al.
1 citations
Ultrasound in Obstetrics and Gynecology
Ultrasound and Hyperthermia Applications
0.18
article

OC01.04: *AI‐driven ultrasound detection of ovarian cancer that generalises: an international multicentre validation study

Paula Herman, Ekaterini Domali, S. Guerriero, M. Pascual, E. Epstein, Debora Verri, Nina Montik, D. Fischerová, Karina Liuba, Chiara Visenzi, Luca Savelli, L. Haaková, Juan Luis Alcázar, Maria Munaretto, Marek Kudła, Frank Christiansen, Emir Konuk, Joana Palés Huix, Adrius Gaurilčikas, Robert Fruscio, Kevin Smith, Petra Šašková, D. Franchi, Artur Czekierdowski, F. Buonomo, N.P. Pangilinan, F. Leone, Carlo Carella, E Mor, Robert B. Welch, Arul Ganeshan
article en
1 citations

Abstract

A critical shortage of expert ultrasound examiners has raised concerns of unnecessary interventions and delayed diagnoses. Artificial intelligence (AI)-driven diagnostic support has the potential to alleviate this burden and improve patient outcomes. Deep learning has shown promising results in ovarian cancer detection; however, external validation is lacking. Our aim was to develop a deep learning model for ovarian cancer detection and evaluate its robustness and ability to generalise in a large multicentre setting. We retrospectively collected 17,119 ultrasound images from 3,652 women with an ovarian lesion (2,224 benign, 1,428 malignant) from 20 centres in eight countries. A total of 2,718 cases were reviewed by a minimum of 7 expert and 6 non-expert examiners. For each centre in turn, we trained a transformer-based model using data from the remaining centres and compared the models' performance with that of human examiners. Furthermore, we simulated and assessed how these models could be used in AI-assisted triage and compared it to current practice. The models demonstrated robust performance across centres, ultrasound systems, and histological diagnoses, with an area under the receiver operating characteristic curve of 0.93 (95% CI 0.92–0.94) and F1 score of 83.7% (95% CI 82.3–85.0) on cases from unseen centres. They outperformed both expert and non-expert examiners, with F1 scores of 79.6% (95% CI 78.3–81.3) and 74.7% (95% CI 73.0–76.3), respectively. The models were further shown to produce well-calibrated predictions. In a retrospective simulation, AI-assisted diagnostic support reduced the number of referrals to experts by 63% compared to current practice, while increasing the diagnostic performance (F1 83.0% vs 77.5%). The models exhibit strong generalization and outperform expert examiners. Introducing AI-driven diagnostic support into the clinical workflow may reduce human resource demands, while improving diagnostic performance.

Ultrasound in Obstetrics and GynecologyVol. 64(S1)
Medical University of Lublin (PL), Marche Polytechnic University (IT), Medical University of Silesia (PL), University of Cagliari (IT), National and Kapodistrian University of Athens (GR), IRCCS Materno Infantile Burlo Garofolo (IT), Science for Life Laboratory (SE), Karolinska Institutet (SE), Luigi Sacco Hospital (IT), Hospital Universitario Dexeus (ES), Azienda Ospedaliera San Gerardo (IT), Clinica Universidad de Navarra (ES), Skåne University Hospital (SE), Ústav pro Péči o Matku a Dítě (CZ), Fondazione Poliambulanza Istituto Ospedaliero (IT), Istituti di Ricovero e Cura a Carattere Scientifico (IT), European Institute of Oncology (IT), KTH Royal Institute of Technology (SE), University of Bologna (IT)
Università degli Studi di Cagliari, Karolinska Institutet, Universidad de Navarra, Università di Bologna, Science for Life Laboratory
Openalex Percentile: Top 53%
Ultrasound and Hyperthermia Applications
0.18
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.