DeepLabV3+ with GPT: A Framework for Leaf Disease Image to Text Generation

Accurate diagnosis and description of leaf diseases are crucial for effective agricultural management and profit optimization. Traditional methods of disease identification depend heavily on expert knowledge which makes them time-consuming and less scalable. This research introduces a novel comprehensive framework that integrates DeepLabV3+, a state-of-the-art semantic segmentation model, with the Generative Pre-trained Transformer (GPT) to automate the process of leaf disease detection and generate descriptive textual information about the identified diseases directly from images. The DeepLabV3+ model is fine-tuned to precisely segment and classify various types of leaf diseases from high-resolution agricultural images. Subsequently, the segmented images serve as inputs to a GPT-based model trained to generate accurate and contextually relevant descriptions of the diseases. Our experiments demonstrate the effectiveness of this framework in providing detailed, correct, and actionable textual outputs that can assist farmers in making informed decisions. The proposed method enhances the scalability of disease diagnosis and reduces the dependency on expert knowledge, contributing significantly to modern agricultural practices.