Glancing Beyond Patch: Spatial Contextual Cues for 3D Neuron Segmentation

Accurate segmentation of neurons in 3D fluorescence microscopy images is essential for advancing neuroscience. Prevalent methods split a volume into patches and process each patch separately due to computational resource limitations. However, they fail to capture global neuronal morphology across multiple patches, which results in discontinuous segmentation and poses a challenge for subsequent neuronal reconstruction. In this paper, we propose a dual U-Net architecture termed "Glancing Beyond Patch" Network (GBP-Net) to incorporate contextual information into segmentation. Specifically, GBP-Net encodes contextual and high-resolution information using two U-Nets, respectively, and facilitates their integration through a cross-scale context module (CSCM) and a cross-resolution fusion module (CRFM). CSCM utilizes a cross-attention mechanism to enable interaction of features from diverse fields of view after encoders, and CRFM employs Mamba to adaptively fuse high-resolution features after decoders. These two modules complement each other at different levels, enabling to capture both global neuronal structures and fine-grained details. Additionally, the proposed cross-network loss guides to focus on challenging samples by penalizing misclassified voxels from both networks, which further promotes the exploitation of contextual information. Experimental results on three datasets demonstrate that our method outperforms other advanced segmentation methods while maintaining computational efficiency. It keeps the global structure of neurons and achieves the highest F1 scores.