A hybrid feature encoding approach for multimodal aspect-based sentiment analysis

Multimodal aspect-based sentiment analysis (MABSA) plays a crucial role in multimodal sentiment analysis tasks. However, existing methods face challenges in capturing deep semantic interactions between images and text and lack flexible modality fusion mechanisms, limiting the full utilization of multimodal information. To address these issues, this paper proposes a Cross-modal Dynamic Fusion Framework (CDFF), consisting of two modules: the Cross-modal Feature Interaction Module (CMFIM) and the Adaptive Modality Fusion Module (AMFM). Specifically, CMFIM enhances cross-modal semantic associations by repeatedly calculating the semantic similarity between images and text, while AMFM achieves adaptive integration of multimodal information through a dynamic modality fusion mechanism, effectively improving the robustness of the model. Experimental results demonstrate that the proposed method significantly improves MABSA task performance on the Twitter 2015 and Twitter 2017 datasets, outperforming existing methods.