A comprehensive simulation study evaluating the predictive performance of Cox proportional hazards model and machine learning methods for time-to-event data

BACKGROUND: Many data-driven risk prediction models have been developed for analyzing time-to-event data. However, choosing the most suitable model for accurate predictions in a specific medical application remains a challenge. Simulation enables effective comparison on the basis of equal-sized datasets. This study provided a comprehensive and fair comparison of the overall survival prediction performance of machine learning models and the traditional Cox proportional hazards model, using both simulated and real datasets. METHODS: Machine learning models included random survival forests, eXtreme Gradient Boosting, and deep neural networks (DeepSurv). We evaluated the performance of the models using the C-index and the Integrated Brier Score. The evaluation was performed under different data-generating mechanisms, such as varying sample sizes, censoring proportions, the addition of noise variables, and in the presence of different types of model misspecification. RESULTS: All the models showed improved predictive performance with increasing sample sizes. However, their performance declined as the proportion of censoring increased. An increase in noise variables reduced prediction accuracy across all models, regardless of dataset size. Tree-based models demonstrated promising predictive performance compared to the Cox proportional hazards model and DeepSurv in the presence of misspecification and a large number of noise variables. The Cox proportional hazards model performed well with larger sample sizes and fewer noise variables. It also performed well when the model was correctly specified or had only minor misspecification. CONCLUSIONS: Appropriate model for a specific medical application should be motivated by the research question and the underlying data structure.