BASG-Net: a background-aware spatial gating network for cervical cell segmentation in pap smear images

The precise segmentation of cervical cell images is regarded as a fundamental prerequisite for the implementation of computer-aided cervical cancer screening. Pap smear images are often acquired under complex imaging conditions and may be affected by red blood cells, residual staining, mucus, inflammatory background, and cellular debris. These non-cell background components may resemble cervical cells in color, texture, or local morphology, making it difficult for computational models to distinguish true cellular regions from background interference. However, many existing segmentation models mainly rely on feature self-calibration or contextual enhancement, with limited explicit modeling of non-cell/background regions during cross-level feature fusion. To address this issue, a Background-Aware Spatial Gating Network, termed BASG-Net, is proposed in this study for cervical cell segmentation in Pap smear images. Specifically, a background-aware auxiliary branch is embedded within the encoder to estimate a non-cell/background probability map using supervision derived from the inverse binary cell mask. The estimated background response is then converted into a spatial gating signal to modulate multi-level skip-connection features before decoder fusion, aiming to reduce the propagation of background-related responses. Meanwhile, meso-scale convolutions are incorporated into the backbone to enhance the joint representation of local cellular structures and neighboring contextual information. Experimental evaluations on APACS23 and Cx22 provided preliminary evidence for the utility of BASG-Net across different cervical cytology datasets. BASG-Net achieved Dice coefficients of 0.7719 and 0.9425, IoU values of 0.6286 and 0.8913, and accuracies of 0.9796 and 0.9735 on APACS23 and Cx22, respectively. Qualitative comparisons suggested a tendency toward fewer background-related false-positive responses in selected complex smear images. These results suggest that background-aware spatial gating may help improve cervical cell segmentation under complex smear conditions.