A Deep Learning-Based Approach for Cell Segmentation in Phase-Contrast Images

Cell segmentation in Phase-Contrast (PhC) microscopy is crucial for non-invasive analysis of live-cell imaging data. However, traditional segmentation methods often struggle with challenges such as low contrast, overlapping cells, and artifacts that are unique to PhC imaging. In this study, we proposed a deep learning-based model utilizing the Mask Regional Convolutional Neural Network (Mask R-CNN) architecture for segmentation of cells in PhC images. To optimize the performance, three optimizers have been evaluated including Stochastic Gradient Descent (SGD), Adaptive Moment Estimation (Adam), and Ranger. The models have been trained and tested on two PhC datasets, PhC-C2DH-U373 and PhC-C2DL-PSC, derived from the ISBI cell tracking challenge. Models were tested with ResNet50, ResNet101, and ViT-B/16 backbones, and ablation studies assessed the effects of backbone architectures and layer fine-tuning strategies. Results demonstrate that CNN-based backbones with ResNet101 achieved higher performance compared to the ResNet50 and ViT-B/16 backbones. The Ranger optimizer outperformed Adam and SGD with an Intersection over Union (IoU) of 94.3% and an F1 score of 97.1% on the PhC-C2DH-U373 dataset, as well as an IoU of 85.3% and F1 score of 92% on the PhC-C2DL-PSC dataset, both with data augmentation. Statistical validation using the Wilcoxon signed-rank test confirmed the significance of these results (P-Value < 0.01). Furthermore, an IoU improvement of over 10% is observed for densely packed cells as in the second dataset compared to traditional and benchmark approaches. Additionally, we introduced a generalized model by combining the two datasets to increase model reliability and reduce computational complexity. The findings highlight the potential of Ranger and the generalized training model to enhance cell segmentation across different microscopy datasets.