Multimodal learning enables chat-based exploration of single-cell data
Single-cell sequencing characterizes biological samples at unprecedented scale and detail, but data interpretation remains challenging. Here, we present CellWhisperer, an artificial intelligence (AI) model and software tool for chat-based interrogation of gene expression. We establish a multimodal embedding of transcriptomes and their textual annotations, using contrastive learning on 1 million RNA sequencing profiles with AI-curated descriptions. This embedding informs a large language model that answers user-provided questions about cells and genes in natural-language chats. We benchmark CellWhisperer's performance for zero-shot prediction of cell types and other biological annotations and demonstrate its use for biological discovery in a meta-analysis of human embryonic development. We integrate a CellWhisperer chat box with the CELLxGENE browser, allowing users to interactively explore gene expression through a combined graphical and chat interface. In summary, CellWhisperer leverages large community-scale data repositories to connect transcriptomes and text, thereby enabling interactive exploration of single-cell RNA-sequencing data with natural-language chats.
Authors
- Daniel Malzl (ORCID: https://orcid.org/0000-0001-5765-6375)
- Jake Burton
- Mihaela Peycheva (ORCID: https://orcid.org/0000-0001-9268-0637)
- Jörg Menche (ORCID: https://orcid.org/0000-0002-1583-6404)
- Thomas Krausgruber (ORCID: https://orcid.org/0000-0002-1374-0329)
- Salvo Danilo Lombardo (ORCID: https://orcid.org/0000-0001-6723-7019)
- Anna Hakobyan (ORCID: https://orcid.org/0000-0003-4085-701X)
- Celine Sin (ORCID: https://orcid.org/0000-0002-4975-4618)
- Varun Sharma (ORCID: https://orcid.org/0000-0002-4531-640X)
- Christoph Bock (ORCID: https://orcid.org/0000-0001-6091-3088)
- Eleni M. Tomazou (ORCID: https://orcid.org/0000-0002-7497-4567)
- Peter Peneder (ORCID: https://orcid.org/0000-0002-3561-4489)
- Moritz Schaefer (ORCID: https://orcid.org/0000-0001-6489-1947)
Institutions
- University of Vienna (AT)
- Austrian Academy of Sciences (AT)
- Austrian Research Institute for Artificial Intelligence (AT)
- Max Perutz Labs (AT)
- Comprehensive Cancer Center Vienna (AT)
- St Anna Children's Hospital (AT)
- CeMM Research Center for Molecular Medicine (AT)
- St. Anna Children's Cancer Research Institute
- Medical University of Vienna (AT)
Publication Details
- Journal
- Nature Biotechnology
- Published
- 2025-11-11
- DOI
- https://doi.org/10.1038/s41587-025-02857-9
- Citations
- 14
- Primary Topic
- Single-cell and spatial transcriptomics
- Type
- article
- Field-Weighted Citation Impact
- 11.28
Funders
- Vienna Science and Technology Fund
- Austrian Science Fund
- Universität Wien
- Medizinische Universität Wien