An Explainable Multimodal Deep Learning Framework for Early Pancreatic Cancer Prediction from Imaging and Clinical Data

Accurate early prediction and multi-stage diagnostic classification compliant with AJCC TNM staging criteria are essential for effective treatment planning in pancreatic cancer. Although deep learning (DL) approaches show promising results, interpreting their predictions remains challenging. This study proposes Squeeze-and-Excitation Visual-Efficient Tabular Network (SE-VETNet), an interpretable multimodal DL framework designed for early prediction and multi-stage pancreatic cancer diagnostic classification using medical imaging and clinical data. The framework integrates Visual Geometry Group (VGG) and EfficientNet architectures enhanced with Squeeze-and-Excitation (SE) blocks for feature recalibration, alongside Tabular Network (TabNet) for clinical feature extraction. Attention mechanisms identify relevant tumor characteristics and clinical attributes. An optimized processing pipeline preserves image details and clinical data integrity while improving computational efficiency. To address class imbalance, balancing is applied exclusively to training data, while five-fold cross-validation supports robust evaluation and minimizes potential bias. Experiments demonstrate consistent performance across all folds, achieving a mean test accuracy of 99.21% with minimal variability across five diagnostic cohorts (Healthy and T1–T4). Furthermore, Explainable Artificial Intelligence (XAI) techniques, including Gradient-weighted Class Activation Mapping (Grad-CAM), quantitatively validated against expert ground-truth masks via Intersection over Union (IoU), and TabNet attention, provide insights into image-based and clinical feature contributions, respectively. By integrating multimodal learning, robust evaluation, and explainability, SE-VETNet offers a promising approach to early pancreatic cancer prediction, multi-stage staging, and transparent clinical decision-making. However, validation using independent and diverse clinical datasets remains necessary to establish generalizability and assess its suitability for practical deployment.