Exploring the Impact of Dataset Content on Self-Supervised Pretext Rotation Learning: A Comparative Analysis using MobileNet

A significant challenge in self-supervised learning is understanding how the content of a dataset impacts the effectiveness of pretext tasks. This paper investigates the critical role of dataset content in the performance of self-supervised learning techniques, specifically through a comparative analysis using the Pretext Image Rotation task. We observe the effectiveness of an already trained MobileNet model on two entirely distinct datasets, one satellite imagery used for remote sensing purposes, xBD and the other dataset of cat images. Our findings show that the pre-trained MobileNetV2 model, trained on cat images shows better results than when trained on xBD images. The difference in the results can be because cat images contain consistent, upright objects, making rotation recognition more straightforward. In contrast, the diverse and less structured content in xBD, featuring landscapes and buildings, presents a more challenging task for rotation recognition. This paper emphasizes the significance of aligning pretext tasks with dataset characteristics. It highlights the impact of dataset diversity, data quality, and content relevance on the effectiveness of pretext techniques in self-supervised methods. The results emphasize the significance of carefully choosing the dataset in evaluating pretext tasks and their implications for downstream applications.