Temporal Dynamics in Visual Data: Analyzing the Impact of Time on Classification Accuracy

Visual datasets are generally constructed from the samples available at the time of their collection and are not further updated. However, these static datasets do not reflect the distribution changes that occur in real data. We analyze how different collection times lead to a shift in class distribution by collecting a set of Flickr images published over 14 years. The proposed “Visual Classes through Time” (VCT-107) dataset contains images tagged by their publication date and includes 107 classes covering various topics (human-made objects, animals, plants, food, etc.). Images from each class are divided into five collection periods to study the impact of time on classification accuracy. When training different classification models using linear probing, we observe an accuracy loss when training on data from one period and testing on other periods. This happens even in the case of a strongly pre-trained model like DinoV2 ViT-B/14. Intuitively, the performance loss is generally more significant when the collection periods between the training and test data are further apart. Our analysis reveals that the temporal shift varies between classes, with the largest shifts observed for human-made objects and the smallest for natural concepts such as animal species. Our results stress the importance of regularly updating models to adapt to timeinduced changes in the distribution of visual classes, even when using a strongly pre-trained model. We release the VCT-107 dataset to facilitate research on temporal shifts.