Agridoctor: A Multimodal Intelligent Assistant for Agriculture

Accurate crop disease diagnosis is critical for sustainable agriculture and global food security. Existing unimodal methods, such as image-based classifiers and detectors, struggle to integrate agricultural knowledge and support interactive understanding. Recent advances in large language models (LLMs) and vision-language models (LVLMs) enable multimodal reasoning, but their effectiveness in agriculture is constrained by the lack of specialized datasets and domain adaptation. We propose AgriDoctor, a modular multimodal framework for intelligent crop disease diagnosis and knowledge interaction. AgriDoctor introduces agent-based multimodal reasoning into agriculture, integrating a router, classifier, detector, knowledge retriever, and LLMs. To support training and evaluation, we build AgriMM, a benchmark with 400,000 disease images, 831 expert knowledge entries, and 300,000 bilingual prompts. Experiments show that AgriDoctor trained on AgriMM outperforms state-of-the-art LVLMs on fine-grained agricultural tasks, offering a new paradigm for sustainable farming applications.