A visual–omics foundation model to bridge histopathology with spatial transcriptomics
Artificial intelligence has revolutionized computational biology. Recent developments in omics technologies, including single-cell RNA sequencing and spatial transcriptomics, provide detailed genomic data alongside tissue histology. However, current computational models focus on either omics or image analysis, lacking their integration. To address this, we developed OmiCLIP, a visual-omics foundation model linking hematoxylin and eosin images and transcriptomics using tissue patches from Visium data. We transformed transcriptomic data into 'sentences' by concatenating top-expressed gene symbols from each patch. We curated a dataset of 2.2 million paired tissue images and transcriptomic data across 32 organs to train OmiCLIP integrating histology and transcriptomics. Building on OmiCLIP, our Loki platform offers five key functions: tissue alignment, annotation via bulk RNA sequencing or marker genes, cell-type decomposition, image-transcriptomics retrieval and spatial transcriptomics gene expression prediction from hematoxylin and eosin-stained images. Compared with 22 state-of-the-art models on 5 simulations, and 19 public and 4 in-house experimental datasets, Loki demonstrated consistent accuracy and robustness.
Authors
- Keith A. Youker (ORCID: https://orcid.org/0000-0003-2535-7973)
- Qin Ma (ORCID: https://orcid.org/0000-0002-3264-8392)
- Shu‐Hsia Chen (ORCID: https://orcid.org/0000-0002-9168-5775)
- Yang Yu (ORCID: https://orcid.org/0000-0002-7605-5732)
- John P. Cooke (ORCID: https://orcid.org/0000-0003-0033-9138)
- Kristopher W. Brannan (ORCID: https://orcid.org/0000-0002-0316-898X)
- Jun‐ichi Abe (ORCID: https://orcid.org/0000-0001-7439-7774)
- Longhou Fang (ORCID: https://orcid.org/0000-0003-1653-5221)
- Hao Cheng (ORCID: https://orcid.org/0000-0003-3661-1519)
- Weiqing Chen (ORCID: https://orcid.org/0000-0003-3539-9210)
- Pengzhi Zhang (ORCID: https://orcid.org/0000-0001-6920-1490)
- Vrutant Shah (ORCID: https://orcid.org/0000-0001-5342-6513)
- Li Lai (ORCID: https://orcid.org/0000-0002-5731-2705)
- Ken Chen (ORCID: https://orcid.org/0000-0003-4013-5279)
- Guangyu Wang (ORCID: https://orcid.org/0000-0003-4803-7200)
- Shengyu Li (ORCID: https://orcid.org/0000-0003-4305-6306)
- Qianqian Song (ORCID: https://orcid.org/0000-0002-4455-5302)
- Tu Tran
- Nhat‐Tu Le (ORCID: https://orcid.org/0000-0001-7746-7328)
- Yiwei Xiao
Institutions
- Houston Methodist (US)
- The University of Texas MD Anderson Cancer Center (US)
- Cornell University (US)
- University of Florida (US)
- The Ohio State University (US)
Publication Details
- Journal
- Nature Methods
- Published
- 2025-05-29
- DOI
- https://doi.org/10.1038/s41592-025-02707-1
- Citations
- 45
- Primary Topic
- Single-cell and spatial transcriptomics
- Type
- article
- Field-Weighted Citation Impact
- 36.27
Funders
- Cancer Prevention and Research Institute of Texas
- National Heart, Lung, and Blood Institute
- National Cancer Institute
- National Institute of General Medical Sciences
- National Institute of Neurological Disorders and Stroke