1231 Foundation models of cell and tissue biology enabled by custom scaled data generation: insights from 1000 lung tumor samples

<h3>Background</h3> While the field has made significant advances in understanding the cancer-immunity cycle and defining patient ‘immunotypes’, predicting clinical success for a therapeutic in a given patient population remains an unsolved problem. Meanwhile, advances in the field of artificial intelligence (AI) have aided the design of molecules to drug known targets, but have not helped identify novel targets and their appropriate patient populations. Furthermore, recent technological advances have enabled high-resolution, multiplexed spatial profiling of cells, proteins, and RNA in the TME, but high dimensionality combined with challenges such as batch effects limit our ability to use traditional supervised methods to analyze this data. <h3>Methods</h3> We have created a platform to generate multimodal data specifically for self-supervised machine learning. To begin, we create Tissue Microarrays(TMAs) from formalin-fixed, paraffin-embedded (FFPE) tissue blocks through a computationally-driven process that optimizes both core selection and core placement. We then generate multimodal spatial data on these TMAs using three platforms: 16-channel multiplex Immunofluorescence (mIF), 1000-plex spatial transcriptomics (NanoString CosMx), and hematoxylin-eosin (H&amp;E) (figure 1). The design of the TMAs enables the generation of large-scale data where patient biology is not confounded with batch effects. We have used this large-scale, multimodal data to train custom transformer models (figure 2). These models take heavily masked data from all modalities as input and must reconstruct mIF images (figure 3). As a result, they learn a unified spatial representation of biological structure. A core innovation in these models is the ability to do zero-shot inference: they can be prompted with counterfactual questions, such as ‘If we increased IFNg in T cells, what would happen to HLA expression on tumor cells?’. <h3>Results</h3> To date, we have generated multimodal data on 1000 non-small cell lung cancer (NSCLC) samples, and trained models on the full dataset. Inspecting the embedding space of one model at the patient level, we find that patient samples separate by known ‘immunotypes’ such as T cell infiltrated/desert, and that more nuanced separation can be teased apart to reveal novel tissue immunotypes. Additionally, using the model’s ability to answer biological therapeutic counterfactuals, we have uncovered potential novel therapeutic targets for increasing CD8 T cell infiltration in the tumor and enhancing the efficacy of T cell killing. <h3>Conclusions</h3> We have crafted a platform to create multimodal data purpose-built for AI. With this data, we have trained large-scale models that can be used for defining patient populations and identifying therapeutic targets.