A Dual-branch Self-supervised Learning Framework with Counterfactual Attention Learning for Histopathology Nuclei Segmentation

Histopathology whole slide image (WSI) analysis is particularly important in cancer diagnosis, as it helps doctors assess the tissue morphology within the tumor microenvironment. However, the abnormal characteristics of nuclei size, shape, and distribution, along with fuzzy boundaries, size variations, and nuclei clustering, increase the difficulty of segmentation. In multi-class segmentation scenarios, the differences between nuclei can sometimes be subtle, further complicating the segmentation task. Additionally, the scarcity of WSI data labels makes it challenging for previous methods to capture key information within complex structures.To address these issues, we propose a dual-branch network combining Convolutional Neural Networks (CNN) and Vision Transformer (ViT), introducing a counterfactual attention learning mechanism to handle complex structures and subtle variations. To overcome the limitation of scarce annotated data, we designed a self-supervised pretraining method based on contrastive learning and masked image modeling specifically for the dual-branch network, leveraging unlabeled data to enhance model performance. Experimental results demonstrate that our proposed Dual-branch SSL Framework with Counterfactual Attention Learning for Histopathology Nuclei Segmentation (DSF-CAL) outperforms existing semantic segmentation networks across multiple datasets.