Standardizing lines of treatment (LOT) in multiple myeloma: Real-world validation of an automated rule-based algorithm

Abstract Background Accurate reconstruction of electronic health record (EHR) treatment sequences is essential for leveraging real-world data (RWD) in clinical research and trial-eligibility platforms in multiple myeloma (MM). However, inconsistencies in RWD treatment documentation and the complexity of line-of-treatment (LOT) assignments in MM hinder reproducibility and limit comparisons against published trials. To address these limitations, we developed HEAL-MM, an automated rule-based algorithm that systematically parses structured medication data to calculate LOTs and segment treatment into standardized, clinically meaningful phases. The algorithm is designed to be interpretable, adaptable to evolving therapeutic frameworks, and compatible with widely adopted EHR data standards. Here, we report its real-world validation against expert-adjudicated treatment sequences to assess concordance, identify edge-case failure points, and support RWD integration. Methods A stratified cohort of 100 MM patients (9,136 structured treatment events) was curated from 114 unique healthcare systems compiled from the multi-institutional HealthTree Foundation harmonized registry. Cohort selection was randomized to ensure coverage across diagnostic eras (pre-2010, 2015–2020, 2022–2025) and treatment exposures, including CAR T-cell therapies, bispecific antibodies, and anti-CD38-based regimens. HEAL-MM sequenced MM regimens chronologically and applied predefined logic rules to identify LOT transitions, the initiation of a new therapeutic course based on drug changes, treatment gaps, and procedural anchors such as stem cell transplant or CAR T. Within each LOT, the algorithm segments and labels therapy into phases, clinically distinct intervals (e.g., induction and maintenance) using rule-based clinical logic. Outputs were benchmarked against expert-labeled adjudications. Concordance was evaluated using concordance index (C-index), stratified by patient, LOT depth and diagnostic era. Results: HEAL-MM demonstrated high concordance with expert-labeled treatment sequences: 0.94 for LOT identification, 0.98 for treatment phase segmentation, and 0.96 for phase label assignment. Achieving complete agreement with all expert annotations in 90% of patients for LOT assignment, 94% for the number of treatment phases identified, 77% for the specific labels applied to each phase, and 75% across all three dimensions simultaneously. Among discordant patients, the absolute difference in final LOT count was 0.5 ± 0.9. Performance was highest in early lines (LOTs 1–3; C-index: 0.97), with seven patients showing discrepancies at this stage. Concordance declined in later lines (LOT ≥4; C-index: 0.87), where 50% of the discordant cases (n=4) were attributable to misclassifications in earlier LOTs, such as medication restarts following >3-month gaps, demonstrating the compounding effect of early errors on longitudinal accuracy. Temporal stratification revealed improved accuracy in post-2020 cases (C-index: 0.94) compared to 2015–2020 cohorts (0.91), reflecting the increasing adoption of structured treatment standards and EHR data quality improvements. To assess HEAL-MM performance at clinically significant junctures, we analyzed treatment transitions that initiated a new LOT or marked a change in phase, achieving C-indices of 0.97 for LOT transitions, 0.97 for treatment phase shifts, and 0.94 for phase label assignment. Conclusions: HEAL-MM provides a validated, scalable solution for standardizing MM treatment structures in RWD environments, as it accurately detects the exact treatment entry of a new LOT 97% of the time. This rule-based framework enables reproducible LOT and phase assignment across heterogeneous EHR systems, supporting real-time cohort identification across institutions and EHR platforms. The algorithm's modular design allows for continuous integration of novel therapeutic categories and evolving treatment logic. While the algorithm maintained high fidelity overall, early-stage misclassifications consistent with increased heterogeneity and documentation ambiguity occasionally propagated downstream, underscoring the compounding effect of upstream errors in longitudinal reconstruction. With its generalizable structure and compatibility with industry-standard data models, HEAL-MM lays the foundation for structured treatment reconstruction in MM across RWD infrastructure and serves as a transferable blueprint for other hematologic malignancies.