Transformer-Based Multi-Scale Fusion for Robust Predicting Microsatellite Instability from Pathological Images

Microsatellite instability (MSI) is a crucial biomarker for guiding the efficacy of immunotherapy and adjuvant chemotherapy, making its detection essential for effective cancer treatment and prognosis. Traditional MSI prediction methods encounter challenges including high costs and limited accuracy under low tumor purity conditions. Recent advancements have explored deep learning for MSI prediction from pathological images, yet these approaches often overlook the multi-scale nature of pathological images and specific pathological features critical for MSI diagnosis. In this study, we proposed MSIscope, a novel Transformer-based method for detecting MSI from pathological images by fusing multi-scale pathological image information. Our approach consists of three key components: 1) ROI selection: we design a region of interest (ROI) selector based on convolutional neural networks and attention mechanisms, selecting tumor regions and important non-tumor regions as our focus; 2) Multi-scale vision expansion and feature extraction: we develop an algorithm that captures a broader view centered on a specified area to obtain a multi-scale field of view. The CTransPath feature extractor is then used to extract features from the image; 3) Multi-scale fusion Transformer: we propose a multi-scale feature aggregator (MS-Transformer) to aggregate contextual features across regions and scales. Our method was experimentally validated on public datasets, achieving an AU-ROC of 0.911 on the TCGA pan-cancer dataset and 0.887 on the TCGA-CRC dataset, surpassing existing methods. Additionally, it maintains high AUROC on datasets with lower tumor purity, outperforming current approaches. These results highlight the potential of MSIscope as an robust method for MSI prediction.