A Coherent Approach-Based Fine-Tuning of Segment Anything Model Plus Watershed Algorithm for Instance Segmentation of Mitochondria in Electron Microscopy Images

Increasingly available ultrastructural data from a continuously growing diversity of experimental conditions are driving new opportunities for fruitful neuroscientific hypotheses tested in intracellular compartments such as the nanoscale roles of, e.g., the mitochondria. Reliable morphological statistics are based on achieving highly accurate semantic segmentations of EM images. The state-of-the-art deep CNNs can be somewhat brittle; they tend to provide coarse and high-frequency-oscillatory solutions with discontinuities and false positives even for simple mitochondria segmentation. Historically, the current state-of-the-art in medical image segmentation would involve some variant of the encoder-decoder architecture, such as the U-Net architecture. The SAM does not perform as well, since it has not been explicitly trained for the task and does not demonstrate user-interactive, over one billion annotations mostly for natural images. However, the SAM may be applied to segment anything, including medical image segmentation challenging new datasets. This work is aimed at the difficult task of implementing domain adaptation in mitochondria segmentation within EM images obtained from various tissues and species, using deep learning. We do a systematic study to assess SAM’s ability to perform segmentation in medical images, measure its performance on volumetric EM datasets, and show that it is powerful at segmenting instances even under challenging imaging conditions. We provide a fine-tuning SAM which can be naturally trained by SAM at an exemplary scale, benefiting from a diverse and large dataset over one million image masks in 11 modalities. This model would be able to perform precise segmentation for a wide range of targets under various imaging conditions, at the level of performance of specialized U-Net models, or even better. A visual comparison is shown between our fine-tuning SAM model and U-Net, along with an examination of different watershed post-processing strategies to discriminate between adjacent or conjoined instances. Results from our experiments show that the method suggested is fast, very robust, and accurate, with an imbibed model that has improved learning capability. A comprehensive sensitivity analysis has been carried out, and an ablation study using most popular metrics segmentation evaluation is performed, the results quantified by Dice Similarity Coefficient, Jaccard-Index coefficient, Aggregated Jaccard-Index, Panoptic quality, which confirm the robustness and effectiveness of the introduced modules in enhancing the performance of mitochondria segmentation.