Vision Transformers vs. Convolutional Neural Networks: Benchmarking Across Tasks

Abstract: Vision Transformers (ViTs) have recently emerged as a powerful alternative to traditional Convolutional Neural Networks (CNNs) for computer vision tasks. This research benchmarks ViTs against CNNs across multiple tasks including image classification, object detection, and semantic segmentation. Utilizing standard datasets (ImageNet, COCO, and Cityscapes) and deep learning frameworks (TensorFlow, PyTorch, Hugging Face), we analyze performance using metrics like accuracy, F1-score, mean Intersection over Union (mIoU), and inference speed. We also apply SHAP and LIME for model interpretability, alongside regression and predictive analyses to assess performance trends. Results reveal that while ViTs outperform CNNs in scalability and accuracy on large datasets, CNNs still dominate in computational efficiency and performance on smaller datasets. Keywords: Vision Transformers, Convolutional Neural Networks, Image Classification, Object Detection, Semantic Segmentation, TensorFlow, PyTorch, SHAP, LIME