Crop-weed classification using deep learning: a comparative study of CNNs, vision transformers, and interpretable models

Accurate discrimination between crops and weeds was considered essential for precision agriculture, as it supports targeted interventions and efficient resource management. However, practical deployment remained challenging due to label noise, varying illumination, domain shifts, and diverse imaging conditions. In this study, five deep learning models such as ResNet50, ResNet101, Vision Transformer (ViT), CLIP, and the prototype-based PIPNet were evaluated for crop-weed classification using two datasets that differed in acquisition conditions and annotation quality. The results showed a strong influence of data quality on model performance. Models trained on clean, well separated images consistently outperformed those trained on field-collected data, with improvements of approximately 25-30% across architectures. Cross-dataset evaluation revealed a marked decline in performance, with accuracy dropping by up to 50%, indicating limited generalization under domain shift. Among the evaluated approaches, CLIP and PIPNet maintained relatively stable performance and showed greater robustness to common image corruptions. These models appeared to benefit from semantically meaningful and structured feature representations. PIPNet achieved the highest accuracy on the clean dataset while also providing interpretable predictions through learned visual prototypes. Overall, the findings highlighted the importance of high-quality annotations and the need for models that can adapt to changing field conditions. Approaches that incorporate semantic understanding and interpretability showed greater potential for reliable crop-weed classification in practical agricultural environments.