LitGene: an interpretable transformer model integrating text and ontology for gene representation learning

Abstract Representation learning in genomics often relies on sequence, expression, and network data, thereby overlooking a wealth of insights in the scientific literature. Here, we present LitGene, an interpretable transformer-based model that refines gene representations by integrating unstructured textual information with Gene Ontology annotations through contrastive learning. In evaluations across eight benchmark tasks for protein properties, LitGene achieves competitive accuracy and exhibits zero-shot prediction of GO terms, KEGG pathways, and gene-disease associations. LitGene identified ANGPTL6 as a potential candidate gene for idiopathic pulmonary fibrosis (IPF), supported by its differential expression in independent IPF samples, illustrating LitGene’s utility for hypothesis generation. A SHAP-based interpretability tool highlights text features influencing predictions, while an automated literature-grounding framework provides corroborating support where available. Our results suggest that integrating textual and genetic information may help reduce biases associated with under-studied genes. The model is released as open-source code, along with a publicly accessible web interface at https://litgene.cs.unm.edu/ , which allows users to submit gene names, disease names, drug names, or custom text (e.g., publication abstracts) to obtain evidence-based predictions of related biomedical entities.