End-to-End Multimodal Multiple Instance Learning for Cancer Histopathology Classification with Dual-Attention Fusion

In recent years, computational pathology has advanced toward integrating histopathological images and genomic data to improve diagnostic accuracy and biological interpretability. In this study, we propose a lightweight end-to-end multi-instance learning model that integrates whole slide images (WSIs) with gene expression profiles. The proposed method efficiently extracts image features using a reduced MobileNetV4 and treats gene information at the gene set level based on Gene Set Enrichment Analysis (GSEA) to capture functional relationships among genes. Image and gene features are aggregated within and across modalities through self-attention and cross-attention mechanisms. Our experiments on three cancer types, namely low-grade glioma (LGG), non-small cell lung cancer (NSCLC), and breast cancer (BRCA) showed that the proposed model outperformed image-only models and conventional single-vector gene input models. In particular, the ROC-AUC reached 0.740 for LGG, 0.982 for NSCLC, and 0.966 for BRCA, with a notable improvement in PR-AUC observed for BRCA classification. These results indicate that the integration of morphological and molecular information is effective for capturing disease characteristics. Furthermore, the proposed model maintains high classification performance while reducing computational resources, suggesting its potential applicability to large-scale pathological datasets and clinical applications.