Beyond benchmarking: an expert-guided consensus approach to spatially aware clustering

Spatial omics technologies have revolutionized the study of tissue architecture and cellular heterogeneity by integrating molecular profiles with spatial localization. In spatially resolved transcriptomics, delineating higher-order anatomical structures is critical for understanding how cellular organization affects tissue and organ function. Since 2020, more than 50 spatially aware clustering (SAC) methods have been developed for this purpose. However, the reliability of current benchmarks is undermined by their narrow focus on Visium and brain tissue datasets, as well as incorrect interpretation of manual annotation as ground truth. Here, we present SACCELERATOR, a community-driven, extensible framework that standardizes data formatting, method integration, and metric evaluation, and is designed to rapidly incorporate new methods and datasets. SACCELERATOR currently includes 22 SAC methods applied to 15 datasets spanning 9 technologies and diverse tissue types. Our analysis revealed substantial limitations in the generalizability and reproducibility of SAC methods across tissues and platforms. We also demonstrate that anatomical labels commonly used as ground truths are often biased, potentially error-prone, and, in some cases, unsuitable for benchmarking efforts. Rather than scoring and comparing methods, we propose a consensus-guided workflow that aggregates clustering results to generate consensus representations. Descriptive spatial metrics highlight areas of high entropy where method disagreement is highest, enabling targeted feedback for tissue experts. Applied to brain and cancer datasets, this approach uncovered biologically meaningful patterns overlooked by individual methods and manual annotations. Our results underscore the need for iterative, expert-in-the-loop analysis and reveal that traditional evaluation metrics do not always capture the subjective qualities of results. By improving tissue annotation and addressing key benchmarking limitations, SACCELERATOR provides a robust foundation for advancing spatial omics research.

Authors

Institutions

Publication Details

Journal
Nature Methods
Published
2026-08-24
DOI
https://doi.org/10.1038/s41592-026-03194-8
Citations
3
Primary Topic
Single-cell and spatial transcriptomics
Type
article
Field-Weighted Citation Impact
6.24

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
article

Beyond benchmarking: an expert-guided consensus approach to spatially aware clustering

Shahul Alam, Raphaël Gottardo, Ahmed Mahfouz, Siao-Han Wong et al.
3 citations
Nature Methods
Single-cell and spatial transcriptomics
6.24
article

Beyond benchmarking: an expert-guided consensus approach to spatially aware clustering

Shahul Alam, Raphaël Gottardo, Ahmed Mahfouz, Siao-Han Wong, Peiying Cai, Alexander Kanitz, Divya Sitani, Kirti Biharie, Rasool Saghaleyni, Mark D. Robinson, Florian Heyl, Thomas Chartrand, Lena Perry, Brian Long, Anastasiia Okhtienko, Sven Twardziok, Paul Kießling, Roland Eils, Marco Varrone, Jieran Sun, Naveed Ishaque, Teresa Zulueta-Coarasa, Marcel Reinders, Meghan A. Turner, Sebastian Tiesmeyer, Samuel Gunz, Qirong Mao, Lucie Pfeiferová, Niklas Müller‐Bötticher, Sameesh Kher, Christoph Kuppe, Zaira Seferbekova, Amin El‐Heliebi, Søren Helweg Dam, Martin Zacharias, Giorgia Moranzoni, Martin Emons, Daryna Pikulska, Maria Calleja, Sarusan Kathirchelvan, George Gavriilidis, Fadhl Alakwaa, SpaceHack 2.0 participants, Estella Y. Dong, Mar M. Moreno, Nigel S. Chou, Yuzhou Chang, Michael Fletcher, Francesca A. Luongo, Charlotte Soneson, Louis Kümmerle, Vipul Singhal, Shyam Prabhakar, Liya Zaygerman
article en
3 citations

Abstract

Spatial omics technologies have revolutionized the study of tissue architecture and cellular heterogeneity by integrating molecular profiles with spatial localization. In spatially resolved transcriptomics, delineating higher-order anatomical structures is critical for understanding how cellular organization affects tissue and organ function. Since 2020, more than 50 spatially aware clustering (SAC) methods have been developed for this purpose. However, the reliability of current benchmarks is undermined by their narrow focus on Visium and brain tissue datasets, as well as incorrect interpretation of manual annotation as ground truth. Here, we present SACCELERATOR, a community-driven, extensible framework that standardizes data formatting, method integration, and metric evaluation, and is designed to rapidly incorporate new methods and datasets. SACCELERATOR currently includes 22 SAC methods applied to 15 datasets spanning 9 technologies and diverse tissue types. Our analysis revealed substantial limitations in the generalizability and reproducibility of SAC methods across tissues and platforms. We also demonstrate that anatomical labels commonly used as ground truths are often biased, potentially error-prone, and, in some cases, unsuitable for benchmarking efforts. Rather than scoring and comparing methods, we propose a consensus-guided workflow that aggregates clustering results to generate consensus representations. Descriptive spatial metrics highlight areas of high entropy where method disagreement is highest, enabling targeted feedback for tissue experts. Applied to brain and cancer datasets, this approach uncovered biologically meaningful patterns overlooked by individual methods and manual annotations. Our results underscore the need for iterative, expert-in-the-loop analysis and reveal that traditional evaluation metrics do not always capture the subjective qualities of results. By improving tissue annotation and addressing key benchmarking limitations, SACCELERATOR provides a robust foundation for advancing spatial omics research.

Nature Methods
Human Genome Sciences (United States) (US), Agency for Science, Technology and Research (SG), University of Copenhagen (DK), SIB Swiss Institute of Bioinformatics (CH), European Bioinformatics Institute (GB), Allen Institute for Brain Science (US), German Cancer Research Center (DE), University of Basel (CH), Medical University of Graz (AT), University of Zurich (CH), Heidelberg University (DE), Hertie School (DE), University of Michigan (US), Leiden University Medical Center (NL), University Hospital Heidelberg (DE), Centre Hospitalier Universitaire Vaudois (CH), Helmholtz Zentrum München (DE), ETH Zurich (CH), LEO Foundation (DK), Nature And Biodiversity Conservation Union (DE), German Center for Lung Research (DE), Centre for Research and Technology Hellas (GR), Asklepios Klinik Langen (DE), Czech Academy of Sciences, Institute of Molecular Genetics (CZ), Berlin Institute of Health at Charité - Universitätsmedizin Berlin (DE), Allen Institute (US), National Computational Infrastructure (AU), Swiss Cancer Center Léman (CH), University Medical Centre Mannheim (DE), Allen Institute for Neural Dynamics, Centro de Investigación Médica Aplicada (ES), The Ohio State University (US), Technical University of Munich (DE), Genome Institute of Singapore (SG), Chalmers University of Technology (SE), Carnegie Mellon University (US), Friedrich Miescher Institute (CH), Universidad de Navarra (ES), RWTH Aachen University (DE), University of Chemistry and Technology, Prague (CZ), Actelion (Switzerland) (CH), Technical University of Denmark (DK), University of Lausanne (CH), Delft University of Technology (NL)
National Science Foundation, Deutsche Forschungsgemeinschaft, Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung, Bundesministerium für Bildung und Forschung, Forschungszentrum Jülich, Nederlandse Organisatie voor Wetenschappelijk Onderzoek, Universität Zürich, LEO Fondet, German Network for Bioinformatics Infrastructure, National Institutes of Health
Openalex Percentile: Top 6%
Single-cell and spatial transcriptomics
6.24
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.