Optimizing Visual Transformer and Faster R-CNN Integration for Efficient Object Detection

The integration of Visual Transformers (ViTs) with Faster R-CNN has shown significant promise in computer vision tasks requiring both high accuracy and efficient object detection. However, the computational cost and resource requirements of these models often limit their application in real time, resource-constrained environments. This paper proposes a novel optimization strategy for integrating ViT with Faster R-CNN to enhance both performance and efficiency. We introduce an improved ViT-Tiny backbone with a hybrid attention mechanism, CS-attention, that combines high- and low-frequency attention to better capture local and global features while minimizing computational overhead. Additionally, a pyramid feature network (FPN) is incorporated to enhance multi-scale feature extraction, allowing the model to accurately detect objects at varying scales. Experimental results demonstrate that the optimized model achieves high accuracy and real-time processing capabilities, making it suitable for deployment in industrial and edge computing applications. The proposed approach is validated through extensive experiments, providing a general solution for efficient object detection across various domains.