Real-Time Navigation System for Breast Cancer Surgery with Pre- and Intra-Operative Imaging Using Neural Networks

Breast cancer is one of the most frequently diagnosed malignancies in women worldwide, necessitating precise diagnosis and treatment. Breast-conserving surgery (BCS) is the primary treatment for nonpalpable cases, yet current approaches often lack accuracy due to the absence of real-time 3D imaging during surgery. This limitation impairs surgeons’ ability to visualize tumor locations, compromising outcomes and potentially leading to repeat surgeries with higher risks, undesirable cosmetic results, increased costs, and, in some cases, mastectomy. Thus, there is a critical need for a navigation system to facilitate accurate tumor excision in nonpalpable breast cancer through real-time patient-specific 3D tracking. This study leverages 3D magnetic resonance (MR) and 3D ultrasound (US) imaging, deep neural networks (DNN), finite element (FE) simulation, open-source toolkits, and optical tracking to address this need. The research integrates pre- and intra-operative imaging and tracking in two stages. First, automated breast segmentation is achieved using a deep learning approach on multi-modal MRI data from online repositories. The segmentation results guide the development of patient-specific elastic breast phantoms made from polyvinyl alcohol cryogel (PVA-C), which is used for validation and demonstration in the second stage. The second stage focuses on nonrigidly registering intra-operative 3D US (iUS) navigation with pre-operative MRI (preop-MRI). This involves rigid MRI-US calibration, affine registration with injectable fiducials, and elastic registration using a graphical neural network (GNN)-based displacement estimation to account for natural breast deformability. Validation metrics include dice similarity coefficient (DSC), accuracy, sensitivity, and specificity for segmentation; root mean square error (RMSE) for MRI-US calibration; fiducial and target registration errors (FRE, TRE) for affine registration; and median Euclidean error (MEE) and mean absolute error (MAE) for GNN-based predictions. At the same time, TRE validates the complete navigation system. Therefore, TRE values range between 4.505 mm and 4.611 mm, demonstrating high accuracy. Overall, this study relied on a multidisciplinary approach that combines mechanical engineering, artificial intelligence, modeling, simulation and visualization engineering, biomedical engineering, radiology, and medicine. Its primary focus is on enhancing localization accuracy and minimizing tumor-positive margins.