Towards clinically interpretable machine learning in emergency surgery: feature importance and insights across clinical time points in abdominal pain cases

PURPOSE: Machine learning (ML) may support decision-making for acute abdominal pain (AAP), but limited interpretability hinders adoption. We evaluated a random-forest classifier for predicting urgent abdominal surgery and examined how predictive value and feature importance evolve across clinical timepoints. METHODS: In this retrospective single-center study, we included adults presenting with AAP to the emergency department. The outcome was urgent abdominal surgery within 24 h. Features were grouped into stepwise sets. For each feature set, models were trained in 20 random 80/20 train-test splits with randomized hyperparameter search. Performance was summarized by AUC ROC and AUC PR, and interpretability by permutation importance and SHapley Additive exPlanations (SHAP). RESULTS: Among 1,350 patients (median age 43 years, 682 (50.5%) females), 276 (20.4%) underwent urgent surgery. The final model achieved a median AUC ROC of 0.83. Discrimination increased stepwise from basic data (0.53), symptoms (0.61), pain history (0.66), vital signs (0.68), laboratory values (0.76), and physical examination (0.83). Computed tomography added only marginal improvement (0.83). Feature importance shifted from symptoms to vital signs and subsequently laboratory markers, particularly c-reactive Protein, white blood cell count, and prothrombin time, complemented by guarding. SHAP analyses confirmed these trends by consistently linking abnormal laboratory or clinical values to operative outcomes. CONCLUSION: Interpretable ML enables pre-imaging risk stratification for urgent surgery in AAP. Near-final discrimination is achieved after laboratory testing and physical examination, while computed tomography adds limited value at population level. Timepoint-specific feature contribution may facilitate integration of ML into surgical decision-making.