High-optimized crash severity prediction: a calibrated, leakage-aware ensemble stacking framework for deployment-ready decision making

Abstract Predicting traffic crash severity is essential for emergency management, roadside infrastructure deployment and safety policy. The current machine learning (ML) approaches concentrate on accuracy, while overlooking the operational costs of false negatives (severe crash incidents missed) versus false positives (false alarms) and the associated higher societal cost of false negatives. In this paper, we propose a five-step enhancement process to advance a baseline, reproducible crash severity model into a calibrated, cost-sensitive, and operationally tested decision-making system. Our approach includes first: feature space locking for reproducibility and drift-free feature sets. Second: leakage-aware interaction mining using second-order multiplicative features and separation of training and testing dataset. Third: Bayesian hyper-parameter tuning with Optuna and stability selection using multiple seeds. Fourth: a leakage-aware ensemble stacking of four diverse base learners including Random Forest (RF), Gradient Boosting (GB), Histogram GB (HGB), and cost-sensitive Extra Trees (ET) using out-of-fold meta-learning. Fifth: The calibration metrics were computed on the validation partition using 5-fold out-of-fold cross-validation, so that no sample was used to both fit and evaluate its own calibrated probability. Pre-calibration ECE was 0.0637 (Brier score = 0.0742); following isotonic regression, post-calibration ECE decreased to 0.0004 (Brier score = 0.0648), both comfortably within the target of ECE < 0.03. The approach also introduces a feature-cost mapping framework, which segments the 37-feature space into six comprehensible clusters and explicitly links each cluster to cost-reduction strategies. The approach is validated on a large-scale crash database of 2,263,315 records, showing significant gains across all performance metrics, with a macro F1-score of 97.67% and an AUC-ROC of 99.85%. More importantly, the pipeline achieves a 42% reduction in false negatives for the severe class while reducing false positives for the non-severe class by 36%, resulting in an increase of 54,000 more correctly predicted crashes over the baseline. This study provides a robust, replicable, and actionable framework that bridges the gap between statistical performance of a model and operational constraints for real-time traffic safety systems and emergency dispatch systems.