Adaptive Multimodal Fusion via Attention-Guided Feature Selection for Histopathology Image Classification

This paper presents a framework for adaptive multimodal feature fusion that employs attention-based feature selection mechanisms to enhance the classification of histopathology images. The framework integrates medical images and clinical texts through three core modules: the Unified Feature Processing Module (UFPM) for standardized feature preprocessing, the Cross-Modal Attention Module (CMAM) for facilitating interactions between image and text features, and the Selective Feature Alignment Module (SFAM) for aligning features across different modalities. Experimental results on the Quilt-BCGG and Quilt-Derm4 datasets demonstrate the framework's good classification performance, which improves the accuracy and efficiency of histopathology diagnosis through optimized feature selection and alignment. The link to the code: https://github.com/leibabaya/Attention-guided-Adaptive-Fusion