A proper concordance index for models with crossing hazards

Abstract Concordance indices are among the most popular metrics used for model selection and evaluation in survival analysis. This is due to their clear interpretation and these metrics being proper for survival models where hazards cannot cross, such as proportional hazards models. However, current concordance indices are not guaranteed to be proper for models with crossing hazards, such as stratified proportional hazards models and various machine learning based models. We give a precise characterization of the conditions under which a concordance index is proper, when it orders risk via the predicted hazard rate at the first event time of pairs of individuals. In a series of experiments, we demonstrate that previous concordance indices may prefer incorrect models over the true data‐generating model, whereas ours does not. We also investigate the use of concordance indices as targets for secondary loss terms in deep learning models. Our suggested concordance is easily interpretable and is, therefore, useful as a success metric for survival models.