ROOT: Reasoning Over Multimodal Agricultural Observation Data with Transformers for Effective Plant Anomaly Management

Plant anomalies cause annual crop losses exceeding ${\$}$263 billion globally. Manual surveillance by farmers is impractical at scale, leading to indiscriminate chemical use. We present ROOT for addressing challenges in identifying and treating plant anomalies using multimodal Large Language Models (LLMs). ROOT provides a multimodal natural language interface capable of analyzing visual images, mapped field matrices, and answering farmer questions. We also introduce GARDeN, a benchmark dataset for agricultural reasoning tasks, and evaluate eight state-of-the-art LLMs. We discuss performance-latency-cost tradeoffs for practical deployment. ROOT offers a new approach to empower farmers with Generative AI-driven expert knowledge for sustainable crop management.