Wheat leaf localization and segmentation for yellow rust disease detection in complex natural backgrounds

Wheat yellow rust disease poses a significant threat to global wheat yield and grain quality. Early detection of this disease will help to minimize the loss caused by its effects. Existing models work well on images taken in a controlled environment, whereas a uniform background is placed behind the leaf, but these models fail to produce good results in natural settings. Previous research also involves manual interventions in the pipeline to achieve good classification results such as cropping the images, using uniform backgrounds, etc. These systems are not practical to use in natural environments where there will be a lot of background noise to the image and manual cropping becomes an extra step for the farmer. Moreover, the unavailability of the dataset in which images of leaves are taken in a natural setting became another challenge. In this research, a dataset is curated and leaves are annotated for object detection, object segmentation further the leaves are classified into 3 classes ie healthy, resistant, and susceptible. A novel unsupervised image rotation algorithm is proposed that takes input from YOLOv8 to align the leave in such a way that maximum background can be removed by a rectangular bounding box . Then the comparison between multiple state-of-the-art segmentation models ie. UNET, Segment-Anything (SAM), Segnet, LinkNet, PSPNet, FPN, Deep-Labv3+ (Xception), and DeepLabv3+ (Mo-bileNet) has shown that UNET has outperformed all the other segmentation models with an IOU score of 0.9563. Lastly for classification, the performance of multiple convolution neural networks ie. VGG16, Resnet 101(v2), Xception, Mo-bileNetV2, and Transformer-based models ie. Swin trans-former and MobileVit have been compared. Swin transformer has outperformed the state-of-the-art CNN models with an accuracy of 95.8%. This paper proposes a complete robust pipeline that can be deployed in natural environment and does not need any manual intervention to produce good results. This research shows that good localization of leaves and removal of unwanted background noise at the earliest stage of the pipeline will assist the segmentation model to effectively segment the leaf from the background which will enable classification models to achieve high classification accuracy, even when dealing with very small datasets.