Sentiment analysis based on self-supervised learning cross-modal data augmentation

Sentiment analysis is a technique for identifying and extracting opinions and their corresponding emotional tendencies in a way that acts on multiple data types such as text, audio and video, known as multimodal sentiment analysis. However, in multimodal sentiment analysis, the phenomenon of imbalance in the importance of different modalities and the problem of data scarcity lead to the inability of deep learning based methods to perform sentiment analysis effectively. To this end, this paper proposes a sentiment analysis model based on self-supervised learning cross-modal data augmentation with adaptive modal weight optimisation (SELF-AWMM). On the one hand, a context-based data enhancement method is proposed to mine the intrinsic structure of data using comparative learning so as to enhance the feature representation capability of the model. On the other hand, an adaptive modal weight adjustment algorithm is proposed, which takes into account the probability distribution of the prediction results of each modality and allows the model to flexibly adjust the importance weight of each modality according to the characteristics of the input data, thus achieving more accurate and stable prediction results. Experiments verify the effectiveness of the model in solving the modal imbalance problem on the MOSI, MOSEI and SIMS datasets.