A Two-Stage Framework for Multimodal Aspect-Based Sentiment Analysis Using Cross-modal Translation

In recent years, there has been a growing interest in the task of multimodal aspect-based sentiment analysis (MABSA). The goal of this task is to combine visual and textual information to predict the sentiment polarity of the target mentioned in the text. Existing research mainly focuses on the fusion of multimodal features using attention methods. However, these methods face challenges due to the representation gap between different modalities, and the fusion of different modalities increases the risk of introducing noise in modality interactions. In this paper, we propose a novel two-stage framework with cross-modal translation (TSFCMT) for MABSA. To address the issue of semantic representation gap, we leverage image captions as a bridge between visual and textual modalities. Additionally, to avoid introducing noise in modality interactions during cross-modal fusion, we propose a two-stage module that separately utilizes textual and visual cues for aspect sentiment classification. Furthermore, we introduce an aspect-based decision module to comprehensively integrate aspect sentiment polarities from different modalities. Experimental results on two publicly available multimodal aspect-based sentiment analysis datasets demonstrate that our framework outperforms strong baseline models.