DKP-ADS: Domain knowledge prompt combined with multi-task learning for assessment of foliar disease severity in staple crops

Staple crops are the cornerstone of the food supply but are frequently threatened by plant diseases. Effective disease management, including disease identification and severity assessment, helps to better address these challenges. Currently, methods for disease severity assessment typically rely on calculating the area proportion of disease segmentation regions or using classification networks for severity assessment. However, these methods require large amounts of labeled data and fail to quantify lesion proportions when using classification networks, leading to inaccurate evaluations. To address these issues, we propose an automated framework for disease severity assessment that combines multi-task learning and knowledge-driven large-model segmentation techniques. This framework includes an image information processor, a lesion and leaf segmentation module, and a disease severity assessment module. First, the image information processor utilizes a multi-task learning strategy to analyze input images comprehensively, ensuring a deep understanding of disease characteristics. Second, the lesion and leaf segmentation module employ prompt-driven large-model technology to accurately segment diseased areas and entire leaves, providing detailed visual analysis. Finally, the disease severity assessment module objectively evaluates the severity of the disease based on professional grading standards by calculating lesion area proportions. Additionally, we have developed a comprehensive database of diseased leaf images from major crops, including several task-specific datasets. Experimental results demonstrate that our framework can accurately identify and assess the types and severity of crop diseases, even without extensive labeled data. Codes and data are available at http://dkp-ads.samlab.cn/ .