TC-MNET: Multi-Magnification Network Using Pvt and Convnext for Microscopic Image Segmentation

Pathologists often require the examination of whole slide images (WSI) at various magnifications to accurately classify a tissue's region of interest (ROI) into the correct classes. Deep learning models such as fully convolutional networks (FCN), variants of U-Net, and transformer-based architectures are employed for semantic segmentation. These models have assisted pathologists in accurately categorizing the ROI. However, current deep learning algorithms cannot identify regions of interest that necessitate the incorporation of complex contextual information alongside local patterns in an image. We present a multi-magnification network that integrates ConvNext at higher-magnification and a PVT-based transformer model at higher-magnification. The proposed model retrieves local information and leverages the capabilities of transformers to capture long-range dependencies, thereby acquiring intricate contextual information across an extensive field-of-view (FOV) at higher-magnification. The model was trained employing a multi-magnification loss function with deep supervision techniques. The proposed multi-magnification feature fusion framework surpasses leading segmentation designs, achieving a relative performance improvement of 4.56% in prostate cancer tissue segmentation.