CH-Mits: A Cross-Modal Dataset for User Sentiment Analysis on Chinese Social Media

Multimodal social network user sentiment analysis aims to determine users' emotional polarity (positive or negative) by mining the associations between multiple data types such as images and texts. Existing public datasets are mainly constructed from English social media platforms, while Chinese social media datasets for multimodal user sentiment analysis are extremely scarce. In terms of the posts published by Chinese social media users, it is not rare that the emotional polarity delivered by the image and the textual content is inconsistent. Given such emotional inconsistency between images and texts, how to effectively identify users' true emotion polarity is still challenging. Toward the above issues, in this paper, we firstly construct a Chinese social media dataset CH-Mits for multimodal user sentiment analysis. In order to evaluate the usability of the dataset, we conceive and implement a novel model called PEMNet, and compare it with state-of-the-art models based on the CH-Mits dataset. In the end, we analyze the performance of PEMNet on selected samples with emotional inconsistency between images and texts. The constructed dataset and codes for PEMNet are available at https://github.com/Marblrdumdore/CH-Mits.