Prediction of severe pediatric community-acquired pneumonia using multimodal fusion of chest radiographs and clinical data

BACKGROUND: Community-acquired pneumonia (CAP) is a leading cause of morbidity and mortality in children, particularly in low-income countries. Severe CAP is associated with high mortality risk, making early prediction and intervention critical. Despite advancements in artificial intelligence applications for disease prediction, most models have focused on adults, with limited attention to pediatric populations and the integration of chest radiographs and laboratory tests. METHODS: To develop a predictive model for severe CAP in children by integrating multimodal data, including chest radiographs, laboratory tests, and demographic information. A retrospective cohort of 3,964 pediatric CAP patients was constructed using data from Wuhan Children's Hospital. A predictive model for severe CAP was developed by integrating chest radiographs, laboratory test results, and demographic data. Key hyperparameters, including model architecture, initialization parameters, loss function weighting, down-sampling strategies, and multimodal fusion were systematically optimized to enhance performance. These approaches were compared to determine the optimal method for predicting severe CAP in children. RESULTS: The best-performing unimodal model based solely on chest radiographs achieved a PR-AUC of 16.33 ± 1.98% and a ROC-AUC of 78.28 ± 1.48%. Through multimodal fusion, the optimal fusion model significantly improved performance, achieving a PR-AUC of 46.68 ± 4.50% and a ROC-AUC of 92.91 ± 0.64%. SHapley Additive exPlanations values were used to analyze the contributions of individual modalities to the model's predictions, revealing key predictive factors. Furthermore, the regions activated by the model showed substantial overlap with radiologist-annotated lesion areas on chest radiographs. CONCLUSION: This study highlights the potential of multimodal fusion to enhance the prediction of severe CAP in children by integrating chest radiograph and clinical data. The proposed model provides a foundation for future AI-based tools to support early diagnosis and targeted intervention, potentially reducing mortality associated with pediatric CAP.