Abstract 7450: Leveraging large language models' own knowledge for obtaining and exploring gene embeddings

Abstract Transfer learning is increasingly prevalent in gene expression analysis. Large transformer-based models generate embeddings from genes, which are used to construct machine-learning models for various downstream tasks. Given the high cost of training these large models, recent work has utilized ChatGPT-4, a pre-trained language model, to produce embeddings from manually curated NCBI gene descriptions. However, this method does not leverage the vast pre-trained knowledge of ChatGPT-4. In this work, we propose directly querying ChatGPT-4 about genes and converting its responses into embeddings rather than using static NCBI descriptions. For each gene, we independently asked ChatGPT-4 four specific questions designed to elicit ChatGPT’s knowledge about each gene’s general characteristics, regulatory functions, gene structure, and implications in cancer research. We evaluate our approach by comparing the embeddings it produces with those derived from NCBI descriptions. The comparison suggests, notably, that ChatGPT-4’s knowledge closely aligns with that of NCBI despite differences in presentation. Furthermore, ChatGPT-4 reveals more detailed and comprehensive information about many genes. We also used the ChatGPT-answer-derived gene embeddings to distinguish long and short-range transcription factors, dosage-sensitive and insensitive genes, and bivalent, Lys4-methylated, and non-methylated genes. Results indicate a preference for ChatGPT-answer-derived gene embeddings in downstream applications by machine learning models. Overall, our findings suggest that direct querying of ChatGPT-4 offers clear advantages for generating gene embeddings. Citation Format: Dailin Gan, Siyuan Zhang, Xin Lu, Jun Li. Leveraging large language models' own knowledge for obtaining and exploring gene embeddings [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2025; Part 1 (Regular Abstracts); 2025 Apr 25-30; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2025;85(8_Suppl_1):Abstract nr 7450.