Artificial Intelligence in Breast Cancer Lymph Node Evaluation: Is Neoadjuvant Chemotherapy the Achilles’ Heel?

Context.—: Artificial intelligence (AI) has demonstrated high accuracy in detecting lymph node (LN) metastases in treatment-naïve invasive breast cancer. However, its performance post-neoadjuvant chemotherapy (NACT) remains underexplored, where therapy-related morphologic alterations complicate assessment. Objective.—: To evaluate an AI algorithm (aetherAI) for detecting LN metastases in post-NACT invasive breast cancer. Design.—: This retrospective study included 72 patients (58 [80.5%] post-NACT; 14 [19.5%] treatment-naïve), yielding 526 whole slide images and 1290 LNs. AI-generated LN contours were reviewed and modified, if needed. LN status was independently assessed by 4 pathologists and the AI algorithm. Discordant cases were resolved by consensus/immunohistochemistry to define ground truth, against which AI was compared. Results.—: AI accurately contoured 1089 of 1290 LNs (84.4%). Pathologists showed 94.42% agreement, with 5.58% discordance. Against consensus ground truth, AI achieved 94.06% sensitivity, 90.14% specificity, 91.47% accuracy, and area under the curve (AUC) of 0.92. Performance was superior in treatment-naïve LNs (sensitivity, 98.95%; specificity, 98.08%; AUC, 0.99) compared with post-NACT LNs (sensitivity, 92.71%; specificity, 88.36%; AUC, 0.91) (P < .001). Within the post-NACT group, significantly reduced accuracy was associated with interobserver discordance (P < .001) and smaller metastases (micrometastasis, isolated tumor cells) (P < .001), but not noted in treatment-naïve cases. Most false results were attributable to therapy-induced changes. Conclusions.—: Unlike most prior studies restricted to treatment-naïve LNs, this post-NACT cohort demonstrates that NACT-related alterations reduce AI sensitivity and accuracy. Our findings suggest that incorporating post-NACT nodes into algorithm training will be essential for reliable clinical translation of AI-assisted LN evaluation.