WeedCLR: Weed contrastive learning through visual representations with class-optimized loss in long-tailed datasets

Image classification is a crucial task in modern weed management and crop intervention technologies. However, the limited size, diversity, and balance of existing weed datasets hinder the development of deep learning models for generalizable weed identification. In addition, the expensive labeling requirements of mainstream fully-supervised weed classifiers make them cost- and time-prohibitive to deploy widely, for new weed species, and in site-specific weed management. To address these challenges, this paper proposes a novel method for Weed Contrastive Learning through visual Representations (WeedCLR), that uses class-optimized loss with Von Neumann Entropy of deep representation for weed classification. WeedCLR leverages self-supervised learning to learn rich and robust visual features without any labels and applies a class-optimized loss function to address the class imbalance problem in real-world long-tailed weed datasets. WeedCLR is evaluated on two public weed datasets: CottonWeedID15, containing 15 weed species, and DeepWeeds, containing 8 weed species. WeedCLR achieves an average accuracy improvement of 4.3% on CottonWeedID15 and 5.6% on DeepWeeds over previous methods. It also demonstrates better generalization ability and robustness to different environmental conditions than existing methods without the need for expensive and time-consuming human annotations, which could reduce deployment time from days to hours. This is fundamental in reducing weed impacts, which can grow and spread very rapidly further affecting crops and reducing yield. These significant improvements make WeedCLR an effective tool for weed classification in real-world scenarios and allow for more rapid and widespread deployment of site-specific weed management and crop intervention technologies. • A novel self-supervised weed classification method is introduced. • It improves accuracy in long-tailed datasets using a class-optimized loss. • It discerns visual characteristics crucial for identification without annotation. • It outperforms state-of-the-art models in two public weed datasets. • It holds significant potential for advancing automated weed control systems.