A Comparative Study of CNN and Transformer Models for Vehicle Recognition

Vehicle recognition plays a vital role in intelligent transportation systems (ITS), contributing to traffic flow monitoring, vehicle tracking, and smart city infrastructure. With the rise of deep learning, Convolutional Neural Networks (CNNs) and Transformer-based architectures have become dominant solutions for image classification tasks. In this paper, we conduct a comprehensive comparison of seven representative models—five CNN-based models (VGG16, ResNet50, MobileNetV2, ConvNeXt-Tiny, InceptionNeXt-Tiny) and two Transformer-based models (ViT-Tiny, Swin-Tiny)—on a <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{1 0}$</tex>-class vehicle image dataset. All models are fine-tuned using ImageNet-pretrained weights and trained under a consistent experimental protocol. We evaluate their performance using class-wise metrics and analyze model complexity. Furthermore, we investigate misclassification patterns by generating Grad-CAM visualizations on the Top-5 misclassified images per model. Our findings offer insights into model selection trade-offs between accuracy and efficiency, and highlight key differences in decision-making mechanisms between CNN and Transformer architectures. The results provide a solid reference for future deployment of vision models in real-world ITS applications.