Accuracy of O-RADS System in Differentiating Between Benign and Malignant Adnexal Masses Assessed via External Validation by Inexperienced Gynecologists

<b>Objective:</b> To evaluate the accuracy of the O-RADS system in differentiating between benign and malignant adnexal masses, as assessed by inexperienced gynecologists. <b>Methods:</b> Ten gynecologic residents attended a 20 h training course on the O-RADS system conducted by experienced examiners. Following the training, the residents performed ultrasound examinations on patients admitted with adnexal masses under supervision, recording the data in a database that included videos and still images. The senior author later accessed this ultrasound database and presented the cases offline to ten residents for O-RADS rating, with the raters being blinded to the final diagnosis. The efficacy of the O-RADS system by the residents and inter-observer variability were assessed. <b>Results:</b> A total of 201 adnexal masses meeting the inclusion criteria were evaluated, consisting of 136 (67.7%) benign masses and 65 (32.3%) malignant masses. The diagnostic performance of the O-RADS system showed a sensitivity of 90.8% (95% CI: 82.2-96.2%) and a specificity of 86.8% (95% CI: 80.4-91.8%). Inter-observer variability in scoring was analyzed using multi-rater Fleiss Kappa analysis, yielding Kappa indices of 0.642 (95% CI: 0.641-0.643). The false positive rate was primarily due to the misclassification of solid components in classic benign masses as O-RADS-4 or O-RADS-5. <b>Conclusions:</b> The O-RADS system demonstrates high diagnostic performance in distinguishing benign from malignant adnexal masses, even when used by inexperienced examiners. However, the false positive rate remains relatively high, mainly due to the over-interpretation of solid-appearing components in classic benign lesions. Despite this, inter-observer variability among non-expert raters was substantial. Incorporating O-RADS system training into residency programs is beneficial for inexperienced practitioners. This study could be an educational model for gynecologic residency training for other systems of sonographic features.