Dual-task multi-species network design and training using advanced augmentation techniques

Accurate weed localization in pasture environments is essential for effective site-specific weed management (SSWM). This study evaluates two dual-task convolutional neural networks (CNNs) - a truncated ConvNeXt (tCN) and a truncated UniStemNet (tUSN) - for simultaneous plant segmentation and spraypoint detection across four weed species in south-east Australian pastures. To assess model performance, two testing regimes were used: a pooled test, where all images were randomly split for training and validation, and condition-invariance tests, in which models were trained and evaluated on entirely separate groups of images collected under different environmental conditions. Four data augmentation strategies were compared, including advanced image blending techniques CutMix and MixUp, which had not previously been explored in this domain. In the pooled test, the tUSN model achieved a mean Intersection over Union (mIU) of up to 0.897 for plant segmentation, while the tCN model reached an F1-score of 0.880 for spraypoint detection. Under condition-invariance testing, CutMix augmentation provided the most consistent generalization, with mIU values up to 0.897 for plant segmentation, and F1-scores up to 0.954 for spraypoint detection. HistMatch normalization assisted with model generalization in weaker augmentation setups. These findings demonstrate that advanced augmentation and normalization strategies are critical for achieving robust CNN-based weed detection across diverse real-world field conditions.