Abstract 7426: Leveraging deep learning to enable precision medicine via the IMPROVE benchmarking framework

Abstract The rise of targeted therapies for cancer treatment has fueled the development of algorithms that take the molecular profile (e.g. mutations, transcripts, etc.) of a tumor and predict the effect a drug has on this tumor. Recent advancements in deep learning have enabled the prediction of a cell’s response to untested, potentially novel drugs using molecular measurements [1]. To assess the clinical reliability of these algorithms, we first need to measure their performance across a range of data and model systems (e.g. cell lines, organoids, etc.). Toward this end, we introduce a benchmarking framework that enables the comparison of deep learning models for drug response across both cell line and organoid datasets. We built a fully provenanced, federated data collection pipeline that standardizes gene names, chemical structure information, and recalculates drug response curves for 10 different datasets from 4 different model system types, each accessible via our coderdata Python package (https://pnnl-compbio.github.io/coderdata/) which properly formats and processes the data for machine learning ingest. We integrated this package into the IMPROVE benchmarking framework (https://github.com/JDACS4C-IMPROVE) to evaluate the performance of eight machine learning algorithms that either leverage gene expression data alone [2-5], or with copy number [6, 7], mutation[8], and methylation data[9] added. For each algorithm we calculate the performance when using different data splitting methods as well as within-dataset and cross-dataset performance, including multiple model system types. To date, we have collected over 4, 100, 000 drug-cell datapoints across ∼55, 000 drugs and ∼1600 samples and 4 types of model system. Using our framework we evaluate (1) the effects of proper data splitting in cross-dataset analysis within a single dataset, (2) cross-dataset predictions within model systems, and (3) the ability to predict across different model systems. We are currently extending the organoid data to better enable the benchmarking of tools across the model systems and add algorithms to our benchmarking framework. We plan to use this framework to perform robust assessment of clinical applicability of AI/ML in precision medicine. References: 1. Baptista, et al. PLOS Comput Biol. 2023 Mar 23;19(3):e1010200. 2. Jiang, et al. Brief Bioinform. 2022 May 1;23(3):bbac100. 3. Nguyen, et al. IEEE/ACM Trans Comput Biol Bioinform. 2022;19(1):146-54. 4. Jin, et al. HiDRA: Hierarchical Network for Drug Response Prediction with Attention. J Chem Inf Model. 2021 Aug 23;61(8):3858-67. 5. Ke, et al. In: Advances in Neural Information Processing Systems [Internet]. Curran Associates, Inc.; 2017. 6. Ma, et al, BMC Bioinformatics. 2022 Apr 15;23(4):129. 7. Tang, et al. Sci Rep. 2021 Feb 4;11(1):3128. 8. Li, et al. Brief Bioinform. 2024 May 1;25(3):bbae153. 9. Liu, et al. Bioinformatics. 2020 Dec 30;36(Supplement_2):i911-8. Citation Format: Natasha Koussa, Sara Jones, Jeremy Jacoboson, Yannick Mahlich, M Ryan Weil, Sara J. Gosline. Leveraging deep learning to enable precision medicine via the IMPROVE benchmarking framework [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2025; Part 1 (Regular Abstracts); 2025 Apr 25-30; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2025;85(8_Suppl_1):Abstract nr 7426.