Multimodal AI generates virtual population for tumor microenvironment modeling
The tumor immune microenvironment (TIME) critically impacts cancer progression and immunotherapy response. Multiplex immunofluorescence (mIF) is a powerful imaging modality for deciphering TIME, but its applicability is limited by high cost and low throughput. We propose GigaTIME, a multimodal AI framework for population-scale TIME modeling by bridging cell morphology and states. GigaTIME learns a cross-modal translator to generate virtual mIF images from hematoxylin and eosin (H&E) slides by training on 40 million cells with paired H&E and mIF data across 21 proteins. We applied GigaTIME to 14,256 patients from 51 hospitals and over 1,000 clinics across seven US states in Providence Health, generating 299,376 virtual mIF slides spanning 24 cancer types and 306 subtypes. This virtual population uncovered 1,234 statistically significant associations linking proteins, biomarkers, staging, and survival. Such analyses were previously infeasible due to the scarcity of mIF data. Independent validation on 10,200 TCGA patients further corroborated our findings.
Authors
- Naoto Usuyama (ORCID: https://orcid.org/0000-0003-0888-929X)
- Hoifung Poon (ORCID: https://orcid.org/0000-0002-9067-0918)
- Tristan Naumann (ORCID: https://orcid.org/0000-0003-2150-1747)
- Hanwen Xu (ORCID: https://orcid.org/0000-0003-0361-6647)
- Alexandra Q. Bartlett (ORCID: https://orcid.org/0000-0001-7912-6084)
- Ari Robicsek (ORCID: https://orcid.org/0009-0005-7792-0483)
- Cliff Wong (ORCID: https://orcid.org/0000-0001-7867-090X)
- Carlo Bifulco (ORCID: https://orcid.org/0000-0003-1922-6977)
- Angela Crabtree (ORCID: https://orcid.org/0000-0001-6584-8158)
- Bernard A. Fox (ORCID: https://orcid.org/0000-0002-4452-5947)
- Peniel Argaw
- Jaspreet Bagga
- Brian Piening (ORCID: https://orcid.org/0000-0002-2683-8157)
- 裕二 池谷 (ORCID: https://orcid.org/0000-0002-1704-1744)
- Kevin Matlock (ORCID: https://orcid.org/0000-0003-1658-7427)
- Bill Wright (ORCID: https://orcid.org/0000-0001-9232-4288)
- Sheng Wang (ORCID: https://orcid.org/0000-0003-2611-3559)
- Sheng Zhang (ORCID: https://orcid.org/0000-0003-1732-0011)
- Chanwoo Kim (ORCID: https://orcid.org/0000-0001-9914-313X)
- Jeya Maria Jose Valanarasu
- Racheli Ben Shimol
Institutions
- Microsoft (United States) (US)
- Providence College (US)
- Renton Technical College (US)
- Providence Portland Medical Center (US)
- Research Network (United States) (US)
- Microsoft Research (United Kingdom) (GB)
Publication Details
- Journal
- Cell
- Published
- 2025-12-09
- DOI
- https://doi.org/10.1016/j.cell.2025.11.016
- Citations
- 7
- Primary Topic
- Single-cell and spatial transcriptomics
- Type
- article
- Field-Weighted Citation Impact
- 5.64