Interpretable multitask model for clinical pathology image prediction and interpretation

Deep learning (DL)-based pathological image modelling and analysis approaches offer transformative potential for early cancer diagnostics, yet limited sample sizes and a lack of interpretability often hinder efficient clinical translation. Here, we present the interpretable Multi-Task Digital Pathology Model (iMDPath), an end-to-end, highly explainable multi-task deep learning framework that simultaneously addresses these challenges by integrating data augmentation, diagnostic prediction, and visualization of pathological image features. The iMDPath comprises three modules: Augmentation (iMDPath-Aug), Prediction (iMDPath-Pred), and Visualization (iMDPath-Vis). iMDPath-Aug incorporates a vector-quantized variational autoencoder (VQ-VAE) for enhanced data augmentation, capturing essential pathological features from limited datasets. A Swin Transformer-Based (Swin-B) predictor in the iMDPath-Pred module leverages the augmented data to achieve better performance than patch-level and foundation-model-based encoders such as InceptionV3 and Phikon across six diverse cancer pathology datasets, including gastric, breast, lung, and colorectal cancer. Finally, iMDPath-Vis, a novel visualization module combining the full gradient (FullGrad) and occlusion sensitivity analysis, provides pathologists with actionable insights by highlighting the specific tissue regions driving model predictions. Overall, iMDPath not only surpasses existing methods in diagnostic accuracy, sensitivity, and generalization across these datasets, but also offers a transparent and interpretable AI solution for precision oncology, paving the way for more reliable and efficient clinical decision-making.