Positive Net Benefit of Machine Learning: Predicting Postoperative Pancreatic Fistula After Pancreaticoduodenectomy

Postoperative pancreatic fistula following pancreaticoduodenectomy remains a devastating complication. While machine learning models have been developed to predict fistula formation, none have assessed clinical net benefit. We aim to redevelop and verify the clinical utility of machine learning models in predicting postoperative fistula using Lift and Decision Curve Analysis. The National Surgical Quality Improvement Program Pancreatectomy database (2017-2022) was queried for adults with pancreatic neoplasms, cysts, or pancreatitis undergoing elective pancreaticoduodenectomy. A logistic regression, XGBoost, GBM, LightGBM, Neural Network, and AutoML were developed. Lift analysis assessed model performance compared to random chance and Decision Curve Analysis assessed clinical net benefit. SHapley Additive exPlanations were used for feature interpretation. 1,228 (13.1%) patients developed fistula out of 9,370 included in the study. Patients with fistula were more likely to have no preoperative biliary stent (50.2 vs 37.5%), pancreatic duct <3 mm (39.9 vs 22.4%), benign etiology (28.7 vs 12.5%), and soft pancreatic texture (63.4 vs 28.2%), all p<0.001. Based on performance metrics, Lift, and Decision Curve Analysis, AutoML demonstrated superior performance compared to other models (AUC 0.769, F1 Score 0.412, Brier Score 0.102, Lift 2.23, net benefit 0.06). The most influential factors were gland texture, diabetes, body mass index, preoperative hematocrit, and preoperative chemotherapy. Decision Curve Analysis reveals clinical utility of machine learning models expressed as positive net benefit to predict fistula and implement treatment, and should be applied in addition to standard performance metrics to highlight their utility as an adjunct to clinical decision-making.