Unsupervised domain adaptation with self-training for weed segmentation

Accurate crop and weed segmentation in varied field conditions is crucial for advancing automated weed management but remains challenging. Though promising, convolutional neural networks (CNNs) often experience performance drops when deployed in new field environments due to shifts between training and test data distributions. To address this limitation, we proposed a self-training framework using a teacher–student model that adapts CNNs for diverse agricultural contexts. Our method enhances generalization by co-training the student model on both the source domain and pseudo-labelled target domain generated by the teacher model, with teacher parameters updated via an exponential moving average of the student’s model. The main contributions of this work are as follows: (1) we simplified the self-training procedure by using all target predictions, skipping the selection phase, and applying local dynamic weights (LDW) for target pixels during co-training; (2) we optimized iteration by monitoring covariance fluctuations to avoid pseudo-label overfitting and reduced the impact of false labels; (3) we addressed class imbalance with dynamic class weights (DCW) to give more importance to minority classes; and (4) we formulated a loss function integrating both LDW and DCW into the soft intersection over union (softIoU), enhancing weed segmentation effectiveness. We evaluated our framework with the ROSE challenge dataset across eight adaptations involving varied plants, robots, and growth stages, achieving up to a 0.17 mean IoU improvement over popular methods like CycleGAN. Our approach demonstrated consistent performance across diverse agricultural environments, supporting its use in real-field inference. • We introduced a self-training framework for weed segmentation. • Four novel strategies were developed to enhance the performance. • The method was evaluated on datasets with varying domain gaps. • It is demonstrated generalization across different robots and growth stages. • The experimental code is publicly available for future research.