HybridViT-CAB: a vision transformer and convolutional attention network for precision weed detection in agricultural systems

In this study, we propose a new hybrid deep learning architecture combining Vision Transformers (ViT) with Convolutional Attention Blocks (CAB), specifically designed for automated weed detection in precision agriculture. We introduce a novel coupled approach that combines transformer-based global contextual modelling with channel-wise attention-based local spatial feature extraction to classify images into four classes: broadleaf weeds, grass, soil, and soybean crops. Following training on a balanced dataset of 4795 samples resized to 224 × 224 pixels, the proposed model achieves an overall AUC of 0.97. While the ViT-only configuration achieves the highest classification accuracy (91%), the hybrid ViT-CAB architecture offers superior interpretability via Grad-CAM visualisations and more balanced per-class performance across morphologically similar vegetative categories, making it the preferred choice when explainability and class-balanced prediction are priorities for precision agricultural deployment. This compact model (412 K parameters, 1.57 MB) processes each image in 141 ms per step and is thus practical for real-time use in intelligent weed management systems aiming at sustainable farming mechanisms.