An Image-Text Sentiment Analysis Method for Small Samples Based on Image Captioning

With the wide popularity of personal terminals, people prefer social media to share their lives, which provides a rich source for sentiment analysis methods. However, challenges still exist in small-sample sentiment analysis methods. A sentiment analysis method for Small Samples based on Image Caotion and BERT is proposed. Specifically, the model takes a pre-trained language model as the image description decoder and uses a cross-modal attention mechanism to eliminate the effects of misaligned regions. This can further increase the interaction from image to text. Then, the generated descriptions are coupled with the original text in the dataset. The BERT model is used to extract word vectors and output sentiment analysis results. The COCO dataset is used to train the model for image Captioning, and the MVSA dataset is used for training and evaluation of sentiment analysis. The experiment creates Less Sample Segmentation by randomly selecting samples from the dataset. Accuracy and F1 value are used to compare with baseline models to evaluate the model performance. The results show that the Image Captioning-BERT model has a certain performance improvement in sentiment analysis of image-text pairs with small samples.