CGD-MAE: Clip Distillation-Driven Pre-Training Framework for Vehicle Re-Identification

This work introduces a CLIP-guided Masked Autoencoder (CGD-MAE) pre-training strategy designed to enhance the performance of existing ViT-based architectures for vehicle re-identification (V-ReID). We propose a simple yet effective ViT backbone pre-training approach that prioritizes data quality, quantity, and diversity, leveraging Automobile1M—a novel large-scale curated vehicle dataset derived from publicly available sources. Using hierarchical k-means clustering for automatic data curation, we select one million diverse samples from a large pool of vehicle images, addressing long-tailed distributions and improving backbone performance. Moreover, a modified global-context semantic distillation from large CLIP models further emphasizes the impact of dataset curation. Pre-training CGD-MAE on Automobile1M has proven to be beneficial in enhancing performance in state-of-the-art (SoTA) ViT-based model architectures for V-ReID and various downstream vehicle-centric applications. These results highlight its strong potential as a universal vehicle-specific pre-training strategy, enhancing feature learning and adaptability across a wide range of vehicle-related tasks. Code is available at https://github.com/AlexMaks02/CGD-MAE.