A hierarchical multi-channel multi-modality fusion strategy for integrating digital mammography and synthesized digital breast tomosynthesis for breast mass discrimination

Digital breast tomosynthesis (DBT) combined with full-field digital mammography (FFDM), known as the “combo-mode”, can enhance breast cancer detection and discrimination. However, the DBT is not yet a standard breast cancer screening modality in most hospitals, and the “combo-mode” also doubles the dose exposure to the patient more than FFDM alone. In this study, we synthesized DBT images from FFDM to reduce the extra radiation dose and explored a methodology to effectively integrate multifaceted information from both the synthetic DBT and real FFDM. An improved conditional generative adversarial network (cGAN) network was proposed for generating synthetic DBT with image quality qualified for breast mass discrimination. A novel multiple accuracy metrics scoring (MAMS) strategy was proposed for integrating multichannel and multimodality imaging information within a hierarchical fusion framework. We retrospectively collected 441 patients with both DBT and FFDM from Nanfang Hospital (NFH) and 143 patients with only FFDM from the Second Affiliated Hospital of Guangzhou University of Chinese Medicine (GDHCM), with regions of interest (ROIs) covering the malignant, benign, and normal tissues extracted for model training and validation. The synthesized DBT exhibited satisfactory image quality and comparable discrimination ability with the real DBT on the NFH dataset. The proposed MAMS achieved an accuracy of 80%, 79%, 91%, and AUC of 87%, 85%, and 99%, respectively, for the malignant, benign, and normal tissues on the GDHCM dataset. The experimental results indicate that the findings of this study can help improve breast mass discrimination when only unimodal FFDM is available.