Evaluation of machine learning-based prediction of pregnancy complications and adverse birth outcomes using routine data

Maternal mortality rates in the United States are higher than any other high-income nation, with a disproportionate burden on marginalized groups who often lack access to specialized care. We investigate whether machine learning models have the capacity to predict pregnancy and birth complications using only routinely obtained health information. Using a population-based cohort of 442,760 pregnancies from Kaiser Permanente Northern California occurring between January 1, 2011 and October 31, 2021, we train separate XGBoost prediction models for each to predict seven pregnancy and birth complications in six-week intervals. We analyze performance by comparing to historical studies, examining top features of the model, and assessing clinical utility through a cost-effectiveness analysis. Here we show that only three pregnancy complications (gestational hypertension, gestational diabetes, and pre-eclampsia) are predictable at early intervention points within the first trimester (at 12 weeks, AUC values are 0.789 [0.782, 0.706], 0.800 [0.794, 0.806], and 0.740 [0.734, 0.747] respectively), and demonstrate the cost-effectiveness of model-based interventions with no specialized data. This is compared to the at-birth complications of preterm birth, stillbirth, placental abruption, and small for gestational age at 24-weeks which demonstrated limited interventionable performance (AUCs of 0.683 [0.676, 0.690], 0.667 [0.624, 0.708], 0.622 [0.606, 0.638], and 0.654 [0.647, 0.660] respectively). We validate whether such screening models can transfer to an additional cohort of 6105 pregnancies, and find that transfer must be supplemented with local model tuning. While some predictive models showcase opportunities in prenatal care, this is not uniform across complications and time periods. Our work demonstrates both the potential for, and challenges of, early risk assessment and intervention in personalized perinatal healthcare.