Multi Modal Sentiment Analysis Model Integrating Attention Mechanism

Single-modal sentiment analysis technology cannot deal with multi-modal sentiment analysis directly and effectively, and sentiment analysis for multi-modal data has attracted more and more attention. In this paper, attention mechanism is used to interact all modal information in a more comprehensive way. Following the multi-modal emotion analysis paradigm based on attention mechanism, a multi-modal emotion analysis model integrating attention mechanism is designed. The sentiment analysis model designed in this paper has a hierarchical structure, in which the data input layer includes input text, audio and video. Feature extraction layer includes text feature extraction, audio feature extraction and image feature extraction. The attention features fusion layer maps the extracted three modal features to a common attention space, which is learned through a fully connected neural network and activation function. The gated memory layer uses GRUs to model multimodal timing features as storage units and summarize context information. Sentiment analysis layer realizes information sharing through multi-task learning and sharing presentation layer parameters, thus improving the performance of multiple tasks at the same time.