Abstract 5007: Integrated histology-genomic multimodel deep learning to predict prognosis in non-small-cell lung cancer

Abstract Background: Deep learning (DL) provokes a paradigm shift in deciphering high-dimensional data for various clinical applications, and multimodal DL approaches show significant potential to integrate diverse datasets and enhance prediction accuracy. Despite these advancements, multimodal DL remains underutilized in prognostic assessment, particularly for non-small-cell lung cancer (NSCLC) patients. Methods: We analyzed whole-slide imaging (WSI; approximately 6,000 slides and ∼40,000,000 image tiles), whole exome sequencing (WES), and overall survival (OS) data from 2,366 NSCLC patients sourced from various online databases. A Sequential-Adapted Attention (SeAttn) model was developed to process WSI data, while multiple DL models were trained to analyze WES data and clinical features. Multimodel DL was then used to combine these learned features to predict long-term OS outcomes. The model was further confirmed using a validation (N=100) and a held-out test (N=101) cohort. Results: The SeAttn model exhibited strong predictive power for NSCLC histological subtypes, achieving an area under the curve (AUC) of 0.98 in the test cohort (P<0.0001). SeAttn-derived WSI features, baseline clinical characteristics, as well as WES-based genetic features were used individually to predict OS, yielding concordance indices (C-indices) ranging from 0.55 to 0.67. The combination of these multimodal features using a COX-based DL approach further improved the test cohort C-index to 0.71 (95% CI 0.14, P=0.002), with WSI, mutation gene-sets, and baseline clinical data as the top contributors. Notably, our multimodal DL model accurately predicted prognosis for up to 5 years post-diagnosis (P<0.05 for all time-dependent AUCs within 5 years in both validation and test cohorts), and our model could significantly stratify patients based on their OS (P=0.012 in the test cohort). Additionally, we identified and validated novel prognostic markers, including some genetic mutations, co-deletions, and amplification of two microRNA clusters. Conclusion: Our multimodal deep learning model demonstrates substantial predictive capacity for prognosis in NSCLC patients, providing a robust framework for integrating diverse clinical, genetic, and imaging data to yield accurate survival estimates and uncovering clinically relevant biomarkers to guide risk stratification. Our findings underscore the broader promise of multimodal DL, offering a scalable and data-driven approach that can be adapted for other cancers. Citation Format: Yang Xu, Haimeng Tang, Xue Wu, Qiuxiang Ou. Integrated histology-genomic multimodel deep learning to predict prognosis in non-small-cell lung cancer [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2025; Part 1 (Regular Abstracts); 2025 Apr 25-30; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2025;85(8_Suppl_1):Abstract nr 5007.