Multimodal aspect-based sentiment analysis based on heterogeneous graph

The goal of multimodal aspect-based sentiment analysis (MABSA) is to identify the sentiment polarity associated with each aspect target by analyzing pairs of text and images. Most existing methods overlook the heterogeneity between different modalities and fail to fully integrate the data features of different modalities. In addition, connections between modalities are made in a fully-connected approach, leading to increased noise and reduced model performance. This research introduces a Multimodal Heterogeneous Graph Neural Network (MHGNN) based on local connectivity. MHGNN aims to efficiently integrate features from various modalities while minimizing noise by constructing a multimodal heterogeneous interaction graph and establishing precise local connections between different modal nodes. Specifically, to efficiently integrate the features of different modalities, we construct a heterogeneous graph to connect different modalities. Heterogeneous graph can more effectively represent relationships between various modalities and facilitate the integration of the features of different modalities. Additionally, to reduce the effect of noise, we constructed syntactic dependency trees for textual modals. The syntactic dependency tree represents the local connection relationship between text nodes and aspect nodes. And we established local connections between aspect nodes and image nodes by Token-patch alignment. The local connection reduces the introduction of noise and enhances the model's overall performance. Extensive experiments on three multimodal datasets indicate that MHGNN outperforms existing methods.