OC05.01: *Prospective validation of an end‐to‐end machine learning‐based model for the classification of adnexal masses using ultrasonography

Externally validate a machine learning (ML) model capable of classifying adnexal masses using ultrasonography against the Assessment of Different NEoplasia's in the adneXa (ADNEX) model. In this two-cohort prospective study, women with an adnexal mass underwent a transvaginal ultrasound at Imperial College Healthcare, London, UK (ICH: n = 201, 1113 images) or Asan Medical Centre, Seoul, Republic of Korea (AMC: n = 190, 472 images). Adnexal masses were classified by expert subjective assessment (SA) and the ADNEX model (10% risk of malignancy without CA-125). The reference standard was the diagnosis according to expert SA or histology (where available). The ODS model was developed and validated at ICH using deep learning (MANet) for segmentation and radiomics for classification (ridge regression and Pearson correlation). Dice surface coefficient (DICE) was used to measure segmentation performance and the area under the ROC curve (AUC), F1-score, sensitivity and specificity were used to measure classification performance. The ICH test set had a median age of 49 (IQR 37-63) and 39 malignancies (19.4%). The AMC test set had a median age of 36 (IQR 28-47) and 10 malignancies (5.3%). The segmentation model achieved a DICE score (mean ± SD) of 0.79 ± 0.2 in ICH and 0.87 ± 0.14 in the AMC test set. The ODS model had an F1 score of 0.89 (AUC 0.99), sensitivity of 97% and specificity of 99% in ICH and an F1 score of 0.83 (AUC 1.00), sensitivity of 71% and specificity of 98% in the AMC test set. ADNEX had an F1 score of 0.78 (AUC 0.92), sensitivity of 90% and specificity of 90% in ICH and an F1 score of 0.34 (AUC 0.91), sensitivity of 60% and specificity of 89% in the AMC test set. In diverse target populations the ODS model provides an end-to-end method of adnexal mass segmentation and classification, with a comparable performance to ADNEX. Further prospective external validation of the performance of the ODS model is required to increase generalisation prior to integration into clinical practice.