An Auxiliary Two-Stage Model for Multimodal Aspect-Based Sentiment Analysis Based on Aspect- Face Dual Alignment

To tackle the challenges in multimodal aspect-based sentiment analysis, specifically the inadequate feature fusion of multimodal data due to modality disparities and the absence of emotional orientation in image captions, this paper introduces an auxiliary two-stage model for multimodal aspect-based sentiment analysis based on aspect-face dual alignment. By devising a face-aspect dual alignment module, the model leverages the semantic similarity computation capabilities of the CLIP model to achieve fine-grained dual alignment between aspect and face information. Furthermore, it employs the dynamic routing mechanism of capsule networks to alleviate the issue of overlapping fine-grained features in text. Experimental results demonstrate that our proposed model significantly surpasses existing baseline methods in terms of accuracy and F1 score on the Twitter-15 and Twitter-17 datasets, thereby validating the effectiveness of the face-aspect dual alignment module and the auxiliary two-stage module.