Semiotic-Based Construction of a Large Emotional Image Dataset with Neutral Samples

Image Visual Sentiment Analysis (VSA) requires the availability of large annotated datasets, whose construction presents many challenges. The necessity of gathering a large amount of labeled images contrasts with the rigorous, but lengthy, process required for manual annotation based on psychovisual experiments, and with the automatic gathering of large amounts of data roughly labeled based on the sentiment analysis of the text accompanying the images, like captions, tweets and tags. An additional limitation is the scarcity of high-quality datasets with a neutral class, which forces the images to be classified into emotions even when the observers show no emotional activation. In this work, we present a scalable methodology rooted in semiotics and art theory for the construction of a 3-class (positive, negative and neutral) VSA dataset, enabling the downloading of a desired quantity of images while maintaining labeling coherence and accuracy. Based on the proposed methodology, we introduce and make publicly available a VSA dataset of over 100,000 images. To validate the quality of the dataset, we used it to train several classifiers and compared their performance with those of classifiers trained on other datasets. The results, we got, show that the classifiers trained on the new dataset provide better performance when tested on independent datasets, including those commonly used for psycho-visual experiments.