EP01.21: Performance evaluation study of an artificial intelligence‐based support system for fetal cardiac ultrasound screening

The rate of prenatal diagnosis of congenital heart disease (CHD) is still insufficient due to a large disparity in the skills of examiners. In this study, we have evaluated the performance of our artificial intelligence (AI)-based support system for fetal cardiac ultrasound screening toward clinical application. We enrolled 259 fetal cardiac ultrasound videos of 262 normal cases and 117 videos of 38 CHD cases who were screened in the second trimester. The correct positions of 18 different anatomical substructures were annotated with bounding boxes. The time-series information of these detection results was displayed using barcodes and detection rate graphs. We first assessed the diagnostic accuracy of the automatic normal substructure detection and then conducted a comparative study to evaluate the improvement in the screening performance of examiners resulting from the combined use of our AI system. Multiplicity of statistical analyses was controlled by a fixed sequence procedure. There were 44 unskilled doctors and 6 experts enrolled in this study. The sensitivity for the normal substructure detection was 93.5%, showing superiority to the sensitivity threshold of 80% based on preliminary test results (p < 0.001). Next, the specificity was 95.9%, indicating superiority to the specificity threshold of 80% (p < 0.001). Furthermore, the sensitivity of the unskilled doctor's screening using the AI system was 78.4%, showing superiority over the sensitivity of the examiner alone (p = 0.005). In addition, the specificity was 86.5%, showing superiority to the specificity of the examiner alone (p < 0.001). Our AI system demonstrated sufficient accuracy in the automatic normal substructure detection, and its combined use significantly improved the performance of unskilled doctors in fetal cardiac ultrasound screening.