Multimodal Sentiment Analysis Based on a Hybrid Fusion Strategy of Long Short-Term Memory and Attention Mechanism

With the advent of the information explosion era, multimodal data streams have become more and more widespread. As a result, multimodal sentiment analysis has emerged as a prominent area of research. Nevertheless, due to the disparities and intricacies in information across different modalities, conventional approaches encounter numerous difficulties in performing multimodal sentiment analysis. To tackle these issues, this study presents an innovative approach. The Long Short-Term Memory (LSTM) network is adept at drawing out uuimodal features from sources such as text, audio, and video. These uuimodal features are then merged into biinodal features through a bimodal attention fusion module. This sequence progresses as the combined uuimodal and bimodal features are further amalgamated within a trimodal attention fusion module and a gated unit fusion network module via the LAHFS. resulting in two distinct sets of trimodal features. Each set is assessed independently, ultimately leading to a decision-level fusion that establishes the conclusive analysis outcome. We validated this model on widely recognized multimodal sentiment analysis datasets, CMU-MOSI and CMU-MOSEI. Our experimental results reveal that our model's performance excels over many existing sophisticated multimodal sentiment analysis techniques, showcasing its superior capability in refining the accuracy of sentiment analysis.