3D-FuseNet: attention-guided multi-modal fusion for early-stage pancreatic cancer detection

Pancreatic cancer, recognized as one of the most aggressive and invasive malignancies, has intricate tumor diversity, posing significant challenges for doctors regarding patient therapy and mortality. The need for prompt and precise detection is crucial for improving patient health and reducing the elevated mortality associated with this disease. The manual identification of this condition using computed tomography (CT) images and magnetic resonance imaging (MRI) presents significant challenges. This study presents a modern network framework that combines helpful information from different imaging types (CT, MRI, and PET) along with clinical data to make pancreatic cancer diagnoses accurate. The suggested system uses a 3D Convolutional Neural Network (CNN) to find important features in the data and an autoencoder to get relevant features from the original information. Proposed method uses a three-step approach: first, it extracts features using specific encoders for each type of data; then, it applies an attention mechanism to ensure the components match, and finally combines everything for a complete model. The features extracted from each network are subsequently integrated to improve detection accuracy. Experiments performed on datasets indicate that proposed multi-model approach attains a detection accuracy of 96.4%, a precision of 92.80%, and a recall of 92.68%, substantially surpassing single-modality methods. Results from experiments on a widely accessible dataset show the advantages of the proposed fusion approach over individual models and current state-of-the-art techniques, underscoring its potential to enhance detection and treatment success in pancreatic cancer.