The Limits of Pre-Crash Prediction: A Big Data Analysis of Crash Severity on New Zealand Roads

Road-controlling authorities must decide where to invest in safety before crashes occur, yet many models of crash severity rely on information that exists only after a crash has occurred. This study measures how far severity can be predicted from pre-crash information alone, and how much the crash itself adds. Following design science research and the CRISP-DM process, a reproducible Apache Spark pipeline was built and applied to all 705,609 records of New Zealand's Crash Analysis System. Logistic regression and gradient-boosted trees were trained on 16 pre-crash features, with decision thresholds fixed on a validation partition and results reported once on 206,663 held-out crashes. The boosted trees achieved an ROC-AUC of 0.677 (95% CI [0.672, 0.682]) and identified 69.8% of severe crashes while flagging 47% of all crashes. Re-running the same pipeline with five crash-phase indicators, such as the involvement of a pedestrian, cyclist, or motorcyclist, raised the ROC-AUC to 0.806 and cut the share flagged to 27%, while tuning or removing weak features changed it by 0.004 or less, and two earlier implementations reached the same limit. The limit therefore lies in the information available before a crash, not in the algorithm. Within it, the pipeline identifies the speed environment, lane type, and region as the dominant risk factors, separates regional crash volume from severity, and shortlists 46 sites with more severe crashes than their roads and conditions explain. The paper proposes actions for each phase of Haddon's matrix: before, during, and after the crash.