Abstract LB159: A practical guideline for benchmarking unimodal and multimodal foundation models on spatial transcriptomics data in oncology

Abstract Understanding the molecular and morphological structure of the tumor microenvironment is increasingly approached by the use of foundation models (FMs): large-scale, complex machine learning architectures trained on H&E, spatial omics, and/or molecular data. In parallel, data generation and experimental assays measuring molecular features in their spatial context are growing in size, resolution and scale, often no longer limited to only a few hundred genes, but multiple thousand features measured at subcellular resolution. This combination of scaled data generation and complex, often black-box, models requires well-defined benchmarks and transparent baselines to guide practitioners in deciding which approach(es) will best suit a specific goal with specific data. However, existing benchmarks are designed in a computational driven manner and focus only on a coarse-grained set of labels and genes. Although such benchmarks can give a good indication on overall performance, they often fail to capture different disease types and downstream use-cases which in practice will require distinct data types, outputs, and evaluation metrics. They also fail to capture the higher-level aim of foundation models: enabling novel biological findings. Finally, reproducing benchmark results on internal or novel datasets is often limited by complex computational frameworks. This work provides recommendations for building meaningful benchmarks for foundation models on spatial transcriptomics data, as defined by the combination of (a) meaningful, simple baselines, (b) diverse datasets with respect to indication and data type, and (c) meaningful downstream tasks and interpretable metrics. We begin by providing a complete overview of current benchmarking efforts leveraging spatial transcriptomics data and their design choices as well as limitations. To enhance existing benchmarks, we introduce a new set of tasks and metrics designed on common cell and gene signatures present in oncology, and allow for different processing choices designed for subcellular spatial transcriptomics data. For each task we also define a simple, memory-efficient baseline in order to put the performance of current and upcoming foundation models into perspective. We conclude by discussing useful guidelines for defining future benchmarking datasets for foundation models. The novel tasks and benchmarking frameworks introduced in this work provide a crucial toolset to researchers seeking to accurately assess the evolving landscape of uni- and multimodal foundation models for spatial omics. Citation Format: Anna C. Wahl (née Schaar, Dasha Valter, Lisa Sikkema, Stefan G. Stark, Zelda E. Mariet, Tokuwa Kanno, Ediem Al-Jibury, Tobias Heinen, Liam Gonzalez, Marin G. Scalbert. A practical guideline for benchmarking unimodal and multimodal foundation models on spatial transcriptomics data in oncology [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2026; Part 2 (Late-Breaking, Clinical Trial, and Invited Abstracts); 2026 Apr 17-22; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2026;86(8_Suppl):Abstract nr LB159.