TransCDR: a deep learning model for enhancing the generalizability of cancer drug response prediction through transfer learning and multimodal data fusion for drug representation

Motivation: Accurate and robust drug response prediction is of utmost importance in the realm of precision medicine. Although many models have been developed to utilize the representations of drugs and cancer cell lines for predicting cancer drug responses, their performances can be improved by addressing issues such as insufficient data modality, suboptimal fusion algorithms, and poor generalizability for novel drugs or cell lines. Results: We introduced TransCDR, which uses transfer learning to learn drug representations and fuses multi-modality features of drugs and cell lines by a self-attention mechanism, to predict the IC50 values or sensitive states of drugs on cell lines in an end-to-end manner. We are the first to systematically evaluate the generalization of the cancer drug response (CDR) prediction model to novel (i.e., never-before-seen) scaffolds and cell line clusters. TransCDR shows better generalizability than 7 state-of-the-art deep learning models. TransCDR outperforms its 5 variants that train drug encoders (i.e., RNN and AttentiveFP) from scratch under various scenarios. The most critical contributors among multiple drug notations and omics profiles are Extended Connectivity Fingerprint (ECFP) and genetic mutation. Additionally, the attention-based fusion module further enhances the predictive performance of TransCDR. TransCDR trained on the GDSC dataset achieves good predictive performance on external testing sets: CCLE. Finally, we employ TransCDR to predict missing CDRs on GDSC and drug responses of 7,675 patients to 225 anticancer drugs and verify the prediction results via literature and Gene Set Enrichment Analysis. In summary, TransCDR is a powerful tool with promising drug response prediction prospects