Development and validation of a machine learning prediction model for complicated appendicitis in adults: a multicenter retrospective cohort study

Accurate differentiation between complicated appendicitis (CA) and uncomplicated appendicitis is clinically important in the nonoperative management of acute appendicitis in adults. However, multicenter evidence on risk stratification for CA remains limited. To develop and externally validate a machine learning model integrating routine clinical, laboratory, and computed tomography (CT) information for preoperative risk stratification of CA in adults with acute appendicitis. This multicenter retrospective cohort study included 1573 adults with acute appendicitis from 4 hospitals. A total of 652 patients from center 1 were used for model development and were randomly divided into a training cohort and an internal validation cohort at a 7:3 ratio; 438 patients from center 2 were used as external validation cohort 1; and 483 patients from centers 3 and 4 combined were used as external validation cohort 2. Models were developed using 10 machine learning algorithms and optimized by cross-validation. Discrimination, calibration, and clinical net benefit were assessed using the area under the receiver operating characteristic curve (AUC), Brier score, and decision curve analysis (DCA), respectively. The final model was interpreted using SHapley Additive exPlanations (SHAP), and a web-based online prediction platform was developed. Based only on cross-validation results from the training cohort and the prespecified model selection strategy, the support vector machine model was selected as the final model. The model achieved an AUC of 0.910 (95% CI, 0.840–0.968) in the internal validation cohort, 0.797 (95% CI, 0.743–0.846) in external validation cohort 1, and 0.829 (95% CI, 0.776–0.879) in external validation cohort 2. The model showed good calibration and clinical net benefit in internal validation. Substantial calibration drift was observed in both external validation cohorts; following intercept-only recalibration, the Brier scores decreased from 0.16 to 0.14 and from 0.14 to 0.12, respectively. SHAP analysis showed that fibrinogen, neutrophil-to-lymphocyte ratio, and pain duration were important predictors. The web-based online prediction platform developed from the final model provides individualized risk estimates. The model may support preoperative risk stratification for CA in adults, but the original, unrecalibrated model must not be applied directly to local populations. Local recalibration is mandatory before clinical deployment, and the recalibrated model still requires local validation and prospective evaluation of its clinical impact.

Authors

Institutions

Publication Details

Journal
BMC Medical Informatics and Decision Making
Published
2026-11-13
DOI
https://doi.org/10.1186/s12911-026-03972-6
Primary Topic
Appendicitis Diagnosis and Management
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Development and validation of a machine learning prediction model for complicated appendicitis in adults: a multicenter retrospective cohort study

Zhou Zheng, Xuehu Sun, Maokun Feng, Yiyang Yuan et al.
5 links
BMC Medical Informatics and Decision Making
Appendicitis Diagnosis and Management
article

Development and validation of a machine learning prediction model for complicated appendicitis in adults: a multicenter retrospective cohort study

Zhou Zheng, Xuehu Sun, Maokun Feng, Yiyang Yuan, Kai Song, Yi Shen, Qingsheng Yu, Hui Peng, Hui Feng, Qi Zhang
article en

Abstract

Accurate differentiation between complicated appendicitis (CA) and uncomplicated appendicitis is clinically important in the nonoperative management of acute appendicitis in adults. However, multicenter evidence on risk stratification for CA remains limited. To develop and externally validate a machine learning model integrating routine clinical, laboratory, and computed tomography (CT) information for preoperative risk stratification of CA in adults with acute appendicitis. This multicenter retrospective cohort study included 1573 adults with acute appendicitis from 4 hospitals. A total of 652 patients from center 1 were used for model development and were randomly divided into a training cohort and an internal validation cohort at a 7:3 ratio; 438 patients from center 2 were used as external validation cohort 1; and 483 patients from centers 3 and 4 combined were used as external validation cohort 2. Models were developed using 10 machine learning algorithms and optimized by cross-validation. Discrimination, calibration, and clinical net benefit were assessed using the area under the receiver operating characteristic curve (AUC), Brier score, and decision curve analysis (DCA), respectively. The final model was interpreted using SHapley Additive exPlanations (SHAP), and a web-based online prediction platform was developed. Based only on cross-validation results from the training cohort and the prespecified model selection strategy, the support vector machine model was selected as the final model. The model achieved an AUC of 0.910 (95% CI, 0.840–0.968) in the internal validation cohort, 0.797 (95% CI, 0.743–0.846) in external validation cohort 1, and 0.829 (95% CI, 0.776–0.879) in external validation cohort 2. The model showed good calibration and clinical net benefit in internal validation. Substantial calibration drift was observed in both external validation cohorts; following intercept-only recalibration, the Brier scores decreased from 0.16 to 0.14 and from 0.14 to 0.12, respectively. SHAP analysis showed that fibrinogen, neutrophil-to-lymphocyte ratio, and pain duration were important predictors. The web-based online prediction platform developed from the final model provides individualized risk estimates. The model may support preoperative risk stratification for CA in adults, but the original, unrecalibrated model must not be applied directly to local populations. Local recalibration is mandatory before clinical deployment, and the recalibrated model still requires local validation and prospective evaluation of its clinical impact.

BMC Medical Informatics and Decision Making
Anhui Medical University (CN), Anhui University of Traditional Chinese Medicine (CN), First Affiliated Hospital of Anhui Medical University (CN), Second Affiliated Hospital of Anhui Medical University (CN)
Good health and well-being
Appendicitis Diagnosis and Management
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon