Development and Validation of Novel Machine Learning-Based Risk Scores for Multiple Myeloma: Insights from the Harmony Alliance Big Data Platform

Introduction Traditional risk stratification methods in Multiple Myeloma (MM) rely on clinical parameters and cytogenetic profiles, and often fall short in predictive accuracy. The advent of machine learning (ML) presents an opportunity to enhance risk prediction models by leveraging large datasets and complex variable interactions. Objectives The primary objective was to develop and validate three novel ML-driven prognostic models for newly diagnosed MM (NDMM) patients. These models aim to improve standard scores by incorporating a broader range of variables and a comprehensive statistical approach. Methodology We utilized data from the Harmony MM cohort, comprising 15,581 patients from various sources, including clinical trials within the European Myeloma Network. The core study was based on the same cohorts analyzed originally by the R2-ISS study, including 10,843 patients with NDMM enrolled in 16 clinical trials. An imputation technique was applied to enhance missing data annotation, including some completely missing in the R2-ISS cohort like hemoglobin, which was imputed from recently added trials and real-world registry data from both newly diagnosed and refractory patients (N=4,738). We implemented Random Forest (RF) survival models, focusing on maximizing the discriminative ability while minimizing the number of variables used. Variable importance guided dimensionality reduction was applied to eliminate redundant variables. Validation was conducted using out-of-bag cross-validation within the training set and external validation with Myeloma XI trial data, with accuracy metrics including c-indexes and time-dependent AUCs. Results A model based on the common 20 variables achieved a c-index of 0.666 in the training set and 0.667 in the test set for overall survival (OS) prediction, and 0.619 and 0.627 for progression-free survival (PFS) prediction. A simplified model using six variables (age, hemoglobin, B2-microglobulin, albumin, 1q gain, and 17p deletion) maintained high accuracy, with c-index scores of 0.655 in the training set and 0.664 in the test set for OS, and 0.609 and 0.620 respectively for PFS. Our model excluding cytogenetics achieved c-index scores of 0.637 in the training set and 0.648 in the test set for OS, and 0.596 & 0.614 respectively for PFS. Time-dependent AUCs confirmed the superiority and better reproducibility of the ML approaches compared with the ISS, R-ISS, and R2-ISS benchmarks. Furthermore, the ML models showed better generalization across different patient subsets, including both transplant-eligible and ineligible patients. Then, we modeled OS and PFS outcomes including best treatment response, achieving out-of-bag c-index scores of 0.700 for OS and 0.701 for PFS in a subset of 6,518 patients with sufficient annotation. In order to extract treatment-related effects, we classified the diverse treatments into three main groups: IMID-based (3,078 patients), PI-based (1,139 patients), and PI-IMID-based (1,606 patients). This systematic grouping enabled us to discern treatment-related influences on OS and PFS outcomes, enhancing our understanding of their effectiveness. As a result, upfront treatment, 1q gain, and hemoglobin emerged as key predictors of PFS, underscoring their significance in patient prognosis. Conclusion Our study presents three novel ML prognostic models for NDMM that surpass traditional risk stratification methods. These models are accurate, generalizable, and functional with or without cytogenetic data, enhancing clinical utility. Effective in both transplant-eligible and ineligible patients, they underscore the role of AI in personalized medicine. Incorporating treatment response further enhances model performance. An online tool to calculate the score will be presented at ASH.