A Novel Explainable AI-Based System For Improved Prediction of Breast Cancer Response to Neoadjuvant Chemotherapy

We propose a novel AI-based system for breast cancer (BCa) assessment to predict response to neoadjuvant chemotherapy (NAC) into one of three responses: Partial Response (PR), Complete Response (CR), and Stationary Disease (SD), providing a full insight for medical experts about treatment regimens. The proposed AI-based system integrates machine learning (ML) and deep learning (DL) approaches to incorporate both global and local markers for more accurate prediction. The ML approach, based on a decision tree model, learns patterns from global markers extracted through pathology assessments to determine molecular subtypes. This analysis incorporates four standard tests: ER, PR, HER2, and Ki-67. Additionally, it integrates global radiomics descriptors, including tumor morphology, lesion count, and radiologist assessments for axillary nodes (benign vs. suspicious). In addition to assessing global markers, we employed a pre-trained Vision Transformer (ViT-b16) with a multihead adaptive self-attention mechanism to extract local markers from the Region of Interest (ROI) around the breast tumor. This approach eliminates the need for segmentation, which could impact the accuracy of the local AI model’s prediction. The outputs of both models are fused using a GradientBoosting algorithm to predict the response to NAC. The proposed system was tested on 736 2D images along with their corresponding radiomics and pathological markers (CR = 156, PR = 353, and SD = 227). The developed AI-based system achieved an accuracy of 98.91%. Explainability was enabled through heatmaps which visually highlight areas with high attention for decision-making. These results demonstrate promising potential for AI-based early assessment in BCa management.