A multimodal multipath AI system for assessing PAH after VSD correction on echocardiography and chest radiography images

Developing a novel artificial intelligence (AI) system that can automatically detect pulmonary arterial hypertension (PAH) after correcting the ventricular septal defect (VSD) and to help clinicians make reasonable treatment plans. We analyzed data from 1,316 patients under 1 year old who underwent VSD surgery at Women and Children's Hospital, Qingdao University and Qingdao Central Hospital from January 2017 to December 2023. Pediatric patients were classified into two groups based on postoperative echocardiography and cardiac catheterization results: a normal pulmonary artery pressure group (NG) and PAH after correcting VSD group (CD). We trained and validated a multimodal multipath AI system (MMAI) using echocardiography (Echo) and chest digital radiography (DR) dataset. Dice similarity coefficient (DSC) is used to measure the effectiveness of the model in automatic contour segmentation of images. We assessed the recognition performance of MMAI using the area under the receiver operating characteristic curve (AUC), accuracy, and F1-score through internal and external test sets. The ResNet-50 model demonstrates good performance in automatic cardiac contour segmentation, with DSC values of 0.950 ± 0.017 (Echo) and 0.946 ± 0.020 (DR). Compared to single image types, the multimodal model based on the ResNet-50 model performs better in the binary classification task in the training and validation sets, with both AUC and accuracy exceeding 90%. In multipath detection, the MMAI system performs well in the NG and CD by combining internal and external test set detection, with AUC, accuracy, and F1-score all exceeding 0.9. Our preliminary study developed an MMAI system using Echo and chest DR images, showing potential for assisting in the detection of CD in VSD patients under 1 year of age prior to surgery. Further validation is needed to confirm clinical applicability.