Segment Anything for Microscopy

Accurate segmentation of objects in microscopy images remains a bottleneck for many researchers despite the number of tools developed for this purpose. Here, we present Segment Anything for Microscopy (μSAM), a tool for segmentation and tracking in multidimensional microscopy data. It is based on Segment Anything, a vision foundation model for image segmentation. We extend it by fine-tuning generalist models for light and electron microscopy that clearly improve segmentation quality for a wide range of imaging conditions. We also implement interactive and automatic segmentation in a napari plugin that can speed up diverse segmentation tasks and provides a unified solution for microscopy annotation across different microscopy modalities. Our work constitutes the application of vision foundation models in microscopy, laying the groundwork for solving image analysis tasks in this domain with a small set of powerful deep learning models.

Authors

Institutions

Publication Details

Journal
Nature Methods
Published
2025-02-12
DOI
https://doi.org/10.1038/s41592-024-02580-4
Citations
144
Primary Topic
Cell Image Analysis Techniques
Type
article
Field-Weighted Citation Impact
230.39

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Segment Anything for Microscopy

Paul Hilt, Nabeel Khalid, Constantin Pape, Genevieve Buckley et al.
144 citations
Nature Methods
Cell Image Analysis Techniques
230.39
article

Segment Anything for Microscopy

Paul Hilt, Nabeel Khalid, Constantin Pape, Genevieve Buckley, Sheraz Ahmed, Sagnik Gupta, Vikas Rajashekar, Sushmita Nair, Anwai Archit, Andreas Dengel, Marei Freitag, Luca Freckmann, Carolin Teuber, Sebastian von Haaren
article en
144 citations

Abstract

Accurate segmentation of objects in microscopy images remains a bottleneck for many researchers despite the number of tools developed for this purpose. Here, we present Segment Anything for Microscopy (μSAM), a tool for segmentation and tracking in multidimensional microscopy data. It is based on Segment Anything, a vision foundation model for image segmentation. We extend it by fine-tuning generalist models for light and electron microscopy that clearly improve segmentation quality for a wide range of imaging conditions. We also implement interactive and automatic segmentation in a napari plugin that can speed up diverse segmentation tasks and provides a unified solution for microscopy annotation across different microscopy modalities. Our work constitutes the application of vision foundation models in microscopy, laying the groundwork for solving image analysis tasks in this domain with a small set of powerful deep learning models.

Nature MethodsVol. 22(3)
German Cancer Research Center (DE), Max Delbrück Center (DE), German Research Centre for Artificial Intelligence (DE), Universitätsmedizin Göttingen (DE), Rheinland-Pfälzische Technische Universität Kaiserslautern-Landau (DE), Deutsches Konsortium für Translationale Krebsforschung (DE), Franklin University (US), Monash University (AU), University of Göttingen (DE), Charité - Universitätsmedizin Berlin (DE)
Deutsche Forschungsgemeinschaft, Monash University, Georg-August-Universität Göttingen, Deutsche Krebshilfe, Rheinische Friedrich-Wilhelms-Universität Bonn
Industry, innovation and infrastructure
Openalex Percentile: Top 0%
Cell Image Analysis Techniques
230.39
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.