An Explorative Study on Abstract Images and Visual Representations Learned from Them

Imagine living in a world composed solely of primitive shapes—could you still recognise familiar objects? Recent studies have shown that abstract images—constructed by primitive shapes—can indeed convey visual semantic information to deep learning models. However, representations obtained from such images often fall short compared to those derived from traditional raster images. In this paper, we study the reasons behind this performance gap and investigate how much high-level semantic content can be captured at different abstraction levels. To this end, we introduce the Hierarchical Abstraction Image Dataset (HAID), a novel data collection that comprises abstract images generated from common raster image datasets at multiple levels of abstraction. We then train and evaluate conventional vision systems on HAID across various tasks of classification, segmentation, and object detection, providing a comprehensive study between rasterised and abstract image representations. We also discussed if the abstract image can be considered as a potentially effective format to provide visual semantic information and contribute to the vision tasks. Code and models will be made publicly available.