Analysis of image aesthetics assessment as a positive-unlabelled problem

Image aesthetics assessment (IAA) has been traditionally addressed as a supervised learning problem, where the goal is to accurately predict information related to user opinions, such as the mean opinion score, image ratings, or a binary quality label, usually crafted by using a mean score threshold to label images as highly or lowly aesthetic. Supervised approaches fail to take into account the subjectiveness of this problem, as the idea of aesthetic pleasantness varies among different people and different cultures, thus making the labels extremely noisy. However, the existence of worldwide photographic contests, exhibitions and masters implies that, to a reasonable degree, there is a broader consensus about the quality of very high-quality images and photographs. Furthermore, labelling image data for IAA is a difficult process, as a large amount of non-trivial aesthetic judgements are required for obtaining a large-scale IAA dataset. Therefore, in this work we analyse the potential of positive-unlabelled techniques for solving IAA. We propose techniques for building PU datasets from traditional IAA datasets and from available reference datasets of high-quality images, and test several well-known PU algorithms on these. Our results highlight the potential of PU approaches for IAA, as we obtain results close to the state-of-the-art with much smaller sets of labelled data: in experiments with only 5% of labelled in AVA, we reach accuracy levels only 0.03 points below NIMA, and we reach competent balanced accuracy levels in settings with a very limited amount of labelled data and with very simple models. • We study solving image aesthetic assessment as a positive unlabelled problem. • We make positive unlabelled datasets from known image aesthetic assessment datasets. • We get a balanced accuracy 0.03 points below NIMA with 5% of labelled images in AVA.