Leveraging a Modified Contrastive Language-Image Pre-training Model to Align Images and Text for Generating Remedy Text for Malus Pumila Lamina Images

The increasing threat of leaf diseases to the productivity of precision farming necessitates systematic, logical, and scalable leaf identification methodologies. Conventional plant disease detection approaches are often slow, inefficient, and limited in their applicability, restricting the effective management of leaf diseases. This research work recommends a hybrid multimodal model that uses different modes of activities for leaf disease detection and can integrate image and text data in a single frame to improve the accuracy and proficiency of disease classification. The text data include custom-generated remedy descriptors specifically designed for the proposed model. The latter combines Machine Learning (ML) techniques, such as OTSU thresholding, Gaussian filtering, and modified Contrastive Language-Image Pre-training (mCLIP), to classify diseased leaves and propose suitable remedial actions. The proposed mCLIP model combines image and label data to enhance the effectiveness of multi-class image classification and suitable remedy description generation. Unlike existing multimodal approaches that primarily output text describing image features, the proposed model generates remedy text as the output for specific diseases. This novel approach offers a comprehensive solution for leaf disease detection and renders optimistic results for real-time and automated disease identification in agricultural practices, facilitating early intervention and better crop management. The proposed model obtained an accuracy of 98.1%.