Unified cell segmentation for spatial transcriptomics via transcript reassignment

High-resolution spatial transcriptomics (ST) measures gene expression as individually localized molecules on imaging-based platforms or as counts in subcellular bins on sequencing-based platforms, and both must be assigned to cells before cell-level analysis. Existing segmentation methods focus on delineating cell boundaries, yet transcripts that diffuse beyond their cell of origin, fall where cells overlap, or lie in bins that cells only partially cover are still misassigned. Here we present Tracel, a two-stage optimal transport framework that assigns spatial observations to cells under the guidance of a matched single-cell RNA-sequencing reference. Tracel first aligns reference cells to spatial cells to estimate the expected expression profile at each cell location, and then maps transcripts to cells against these profiles. The same formulation assigns transcripts from imaging-based platforms (e.g., Xenium, CosMx) and bin counts from sequencing-based platforms (e.g., Visium HD). Across five human datasets spanning three platforms and three tissues, Tracel improves the separation of cell types over existing methods, assigning cell-type markers to their correct cells, retaining more cell-intrinsic transcripts and reducing contamination from neighboring cells. Tracel also preserves variation within cell types, yielding more accurate differential expression across spatial locations, and its expression profiles attribute ligand-receptor interactions to the correct cell-type pairs in cell-cell communication analysis. Tracel can enter the segmentation workflow at different stages, either refining an existing segmentation or segmenting cells starting from nucleus segmentation.

Authors

Institutions

Publication Details

Journal
bioRxiv (Cold Spring Harbor Laboratory)
Published
2026-11-04
DOI
https://doi.org/10.64898/2026.11.03.764126
Primary Topic
Single-cell and spatial transcriptomics
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Unified cell segmentation for spatial transcriptomics via transcript reassignment

Hao Chen, Juexin Wang, Junkai Zhang, Xiyao Cheng et al.
bioRxiv (Cold Spring Harbor Laboratory)
Single-cell and spatial transcriptomics
preprint

Unified cell segmentation for spatial transcriptomics via transcript reassignment

Hao Chen, Juexin Wang, Junkai Zhang, Xiyao Cheng, Wei Chen
preprint en

Abstract

High-resolution spatial transcriptomics (ST) measures gene expression as individually localized molecules on imaging-based platforms or as counts in subcellular bins on sequencing-based platforms, and both must be assigned to cells before cell-level analysis. Existing segmentation methods focus on delineating cell boundaries, yet transcripts that diffuse beyond their cell of origin, fall where cells overlap, or lie in bins that cells only partially cover are still misassigned. Here we present Tracel, a two-stage optimal transport framework that assigns spatial observations to cells under the guidance of a matched single-cell RNA-sequencing reference. Tracel first aligns reference cells to spatial cells to estimate the expected expression profile at each cell location, and then maps transcripts to cells against these profiles. The same formulation assigns transcripts from imaging-based platforms (e.g., Xenium, CosMx) and bin counts from sequencing-based platforms (e.g., Visium HD). Across five human datasets spanning three platforms and three tissues, Tracel improves the separation of cell types over existing methods, assigning cell-type markers to their correct cells, retaining more cell-intrinsic transcripts and reducing contamination from neighboring cells. Tracel also preserves variation within cell types, yielding more accurate differential expression across spatial locations, and its expression profiles attribute ligand-receptor interactions to the correct cell-type pairs in cell-cell communication analysis. Tracel can enter the segmentation workflow at different stages, either refining an existing segmentation or segmenting cells starting from nucleus segmentation.

bioRxiv (Cold Spring Harbor Laboratory)
University of Pittsburgh (US), University of Illinois Chicago (US), Indiana University Indianapolis (US)
Single-cell and spatial transcriptomics
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon