Investigating Performance Patterns of Pre-Trained Models for Feature Extraction in Image Classification

Recent advancements in deep learning techniques, such as the introduction of convolutional neural networks (CNNs) and transformers, have led to significant improvements in the accuracy and efficiency of image classification tasks. Despite these advancements, deep learning methods generally require a large amount of annotated data, which can be challenging in domains where labeled data is scarce. Transfer learning strategies have become a promising solution to this issue. This study compares the performance of recent pre-trained neural networks for feature extraction in image classification tasks. Additionally, we analyze performance patterns across models to identify common trends in their classification accuracy and feature extraction capabilities across diverse datasets. We considered 13 different pre-trained models across six image datasets. Our findings indicate that MetaCLIP, EVA-02, and ViT_large achieved the best overall performance across the selected datasets. Furthermore, the experiments also indicate clusters of models that exhibit highly similar behaviors across the datasets. Our results can support model selection for feature extraction in other image classification tasks.