Optimizing Fusion Strategies and Feature Extractors for Improved Multimodal Sentiment Analysis

In multimodal sentiment analysis, the combination of fusion models and feature extractors is crucial, and there is significant room for improvement in feature extractors to achieve higher accuracy. Most existing image feature extractors do not use Attention mechanisms, which have shown great success in text processing. In this paper, I first tested various fusion models on the MVSA-single dataset, and then selected the optimal fusion medels to test different text feature extraction pre-trained models. Next, based on the previous steps, I tested several different image feature extraction pre-trained models. Finally, under the optimal combination of fusion strategy and text feature extractor, I improved a single image model by adding various Attention mechanisms and tested their impact. Based on this, the same improvements were applied to multiple image feature extractors. For different image feature extractors, the accuracy increased by $1 \%$ to $5 \%$ compared to the baseline.