Automated detection and classification of broadleaf and grassy weeds in fallowed fields through vision transformer

In fallowed fields, the presence of broadleaf and grassy weeds poses a significant threat to crop yield and quality if left uncontrolled. Broadleaf weeds, characterized by their wide leaves, and grassy weeds, with their narrow blades, compete vigorously with crops for essential resources such as sunlight, water, and nutrients. Identifying and managing these weed species effectively is paramount for agricultural success. Traditional weed control methods often rely on the use of broad-spectrum herbicides applied across entire fields, regardless of the specific weed composition. This method not only contributes to environmental damage but also incurs unnecessary costs for farmers. In recent years, the Vision Transformers (ViT) have revolutionized the field of Computer Vision, offering unprecedented capabilities in image understanding and analysis. This technique can be applied as a powerful tool to automatically detect and classify both broadleaf and grassy weeds in pre-planting herbicide spraying (known as green-on-brown application). This study aims to develop a system to detect and classify broadleaf and grassy weeds in fallowed fields using a Transformer-based algorithm, YOLOS (You Only Look One Sequence). The dataset comprises 15, 542 images collected from a real fallowed field. Images were splitted into three distinct subsets: training (10, 879 images ≈ 70%), validation (2, 798 images ≈ 18%), and test (1, 865 images ≈ 12%) sets. The model achieved an overall precision of 90.7% (88.3% for broadleaf weeds and 93.0% for grassy weeds) and an average recall of 86.3% (85.3% for broadleaf weeds and 87.2% for grassy weeds). The results suggest that the YOLOS presents a compelling alternative for distinguishing between broadleaf and grassy weeds in fallowed fields.