Multimodal sentiment analysis with interactive gated attention network

Abstract In the field of multimodal sentiment analysis, it is an important research task to fully extract modal features and perform efficient fusion. Traditional sentiment classification models usually have the problems of insufficient interaction information, easy to be affected by visual or perceptual interference, and lack of stability. To address these issues, this paper proposes an interactive gated attention network for multimodal sentiment analysis. Firstly, the feature extraction network based on the pretraining model is constructed to obtain high-dimensional feature vectors. Secondly, the mutual feature vector generator is designed to generate interactive feature vectors with high levels. Furthermore, the gating vector generator is designed to highlight the differences in semantic features. Finally, attention of the interaction mechanism is introduced to focus on the important parts between features, and the feature enhancement and fusion are realized. Comprehensive experiments and analysis on the Twitter-15 and Twitter-17 datasets show that the proposed model is superior to a series of comparative models in information interaction, semantic enhancement, and feature fusion.