Comparative evaluation of Cox regression and machine learning approaches for survival prediction in stage IV lung cancer

Accurate survival forecasting in stage IV lung cancer is essential for making informed therapeutic decisions and utilizing resources more effectively. The Cox Proportional Hazards (CoxPH) model is widely used due to its clear interpretation and strong statistical basis. Nevertheless, new machine-learning (ML) methods, such as Random Survival Forests (RSF), Decision Trees (DT), and Extreme Gradient Boosting (XGBoost), present the capacity to model complex variable interactions and nonlinear relationships. The study compares traditional survival models with machine learning-based to assess how well they predict results and identify key prognostic markers in stage IV lung cancer. Four survival analysis models (Cox Proportional Hazards, Random Survival Forest, Decision Tree, and XGBoost) were used on a clinical dataset of patients with stage IV lung cancer. Model performance was assessed by employing the concordance index (C-index) and the time-dependent area under the ROC curve (AUC). Kaplan–Meier curves were adopted to categorize patients into high- and low-risk cohorts according to model-derived risk scores. The examination of feature importance was performed to specify the main predictors of survival. In the Cox proportional hazards model, radiation therapy, gender, lymph node metastasis, pleural metastasis, and liver metastasis were determined as important predictors (C-index = 0.76). The Kaplan–Meier curves revealed a distinction between high- and low-risk cohorts. The XGBoost model, optimized through hyperparameter tuning, reached a C-index of 0.72 and an AUC of 0.73, emphasizing pleural, lymph-node, brain, and adrenal metastases as key predictors. The RSF model showed the most increased discriminative capability (C-index = 0.76; AUC = 0.78), with liver and pleural metastases, gender, lymph-node involvement, and radiation determined as main factors. The DT model revealed moderate performance (C-index = 0.64; AUC = 0.65) while keeping the benefits of interpretability. The integration of machine learning methods with hyperparameter optimization and systematic feature assessment improves prognosis accuracy in stage IV lung cancer. Although XGBoost and RSF showed improved prediction capabilities, DT and CoxPH presented clinically interpretable insights, highlighting the enduring significance of traditional statistical models. These outcomes support a hybrid analytical approach that integrates performance-oriented machine learning methods with transparent regression models. Validation with extensive, multi-center datasets is critical to enhance generalizability.