Development, validation and interpretability analysis of a machine learning-based model for predicting in-hospital mortality in very low birth weight infants with neonatal respiratory failure

Very low birth weight (VLBW) infants with neonatal respiratory failure (NRF) are at high risk of in-hospital mortality. This study aimed to develop and externally validate a machine learning model for predicting subsequent in-hospital mortality at 24 h after initiation of respiratory support. This multicentre retrospective study included VLBW infants with NRF treated in 26 tertiary NICUs in Jiangsu Province, China, from March 2019 to February 2022. Data from 20 centres were used for development and six centres for external validation. Candidate predictors were restricted to information available before the 24-h prediction landmark. Predictors were selected using Spearman correlation analysis, Boruta and least absolute shrinkage and selection operator regression. Eight algorithms were evaluated using pooled out-of-fold (OOF) predictions generated within a nested cross-validation framework, and the final algorithm was selected according to pooled OOF AUC. SHapley Additive exPlanations (SHAP) was used for model interpretation. The development cohort included 1,294 infants (183 deaths; 14.1%), and the external validation cohort included 239 infants (35 deaths; 14.6%). Ten predictors were retained, and XGBoost was selected as the final algorithm. The model achieved an OOF AUC of 0.751 (95% CI 0.714–0.788) with a Brier score of 0.1050 and an external-validation AUC of 0.754 (95% CI 0.700–0.849) with a Brier score of 0.1148. Calibration was close to ideal in internal validation (intercept, 0.048; slope, 0.959) but deviated from ideal in external validation (intercept, 0.183; slope, 0.774). At the fixed decision threshold of 0.171, external-validation sensitivity was 0.600. Decision curve analysis showed net benefit across a range of threshold probabilities. SHAP identified birth weight and gestational age as the leading contributors to model predictions. The XGBoost model showed similar discrimination in internal and independent external validation for subsequent in-hospital mortality in VLBW infants with NRF who reached the 24-h prediction landmark. Further independent external validation and prospective clinical impact studies are warranted.