Use of artificial intelligence–assistance software for HER2-low and HER2-ultralow IHC interpretation training to improve diagnostic accuracy of pathologists and expand patients' eligibility for HER2-targeted treatment.

1014 Background: The advent of HER2-targeted antibody-drug conjugates and the introduction of HER2-low and HER2-ultralow diagnostic categories have made precise HER2 IHC assessment crucial for optimal breast cancer treatment. However, reproducible and accurate HER2 IHC scoring, particularly in cases with low level of HER2 expression, remains challenging. Many patients with HER2-low or HER2-ultralow expression risk being misclassified as HER2 null, potentially missing access to effective HER2 targeted therapies. Artificial intelligence (AI) assisted HER2 assessment may improve pathologists’ diagnostic accuracy and concordance during interpretation training, especially in challenging cases with minimal membrane staining. Methods: A training platform for AI-supported digital HER2 IHC assessment of breast cancer samples was developed for pathologists. A total of 105 pathologists from 10 countries participated in masterclass sessions, assessing 20 digital HER2 IHC-stained breast cancer cases both without and with AI assistance. Cases assigned ground-truth IHC scores by a central reference center, were divided into three exams: A (n = 5), B (n = 7), and C (n = 8). The masterclasses consisted of: (1) Exam A, (2) a lecture on HER2 IHC scoring, (3) Exam B, (4) discussion of results from Exams A and B, and (5) AI-assisted Exam C. The AI software was used for decision support only for Exam C. The HER2 IHC scoring followed ASCO/CAP 2023 guidelines, adapted to include the HER2-ultralow (IHC 0 with membrane staining) and HER2 null (IHC 0 with no membrane staining), and provided individual tumor cell classifications for explainability. Results: Across 1,940 readings, pathologists achieved an average agreement of 76.3% with reference scores without AI (Exams A+B), compared to 89.6% with AI-assistance (Exam C). For HER2 clinical categories (null, ultralow, low, positive) accuracy improved from 66.7% without AI to 88.5% with AI. Misclassification of HER2-ultralow cases as HER2 null occurred in 29.5% of readings without AI but decreased to 4.0% with AI assistance. Conclusions: AI-assisted training improved pathologists’ accuracy in HER2 IHC scoring by 13.3%, compared to central reference scores. Furthermore, AI reduced the misclassification of HER2-low and HER2-ultralow cases as HER2 null by 25.5%, potentially enabling more patients to access HER2-targeted therapies. These findings highlight the value of AI systems in biomarker interpretation training, providing pathologists with enhanced decision-making tools at the individual cell level and improving diagnostic precision in HER2 IHC interpretation.