Analysing the potential of ChatGPT to support plant disease risk forecasting systems

• We evaluated the potential of ChatGPT to support farmers in plant protection. • Lexical, consistency, and sentiment analyses were part of the evaluation framework. • GPT-3.5 delivers concise, direct advice, ideal for routine, time-sensitive tasks. • GPT-4 provides context-aware messages, suited for strategic, deep reasoning tasks. • Expert review identified the need for domain-specific training prior to deployment. This study explores the potential of two versions of the ChatGPT large language model (GPT-3.5 Turbo and GPT-4) in supporting plant disease risk forecasting through the translation of model-based predictions into advisory messages. A dataset of 3125 messages, each referred to an artificially generated five-day disease risk scenario, was inspected using lexical, consistency, and sentiment analyses. A participatory approach was adopted involving officers and technicians from eleven Italian regional phytosanitary services in the message evaluation. Lexical analysis indicated that GPT-4 produced more detailed advises, leading to diversified responses across disease pressure scenarios. In contrast, GPT-3.5 generated concise and straightforward messages, making it well-suited for routine tasks requiring clarity and brevity. Sentiment analysis revealed the greater adaptability of GPT-4 in shifting from a reassurance to an urgency tone as the risk level increased, while GPT-3.5 maintained a more neutral stance across disease pressure scenarios. Consistency analysis demonstrated greater stability in GPT-3.5 messages, whereas GPT-4 exhibited more expressivity and creativity. Expert evaluations highlighted promising potential of both models for operational use, with GPT-4 noted for its precision in communicating disease risk and supporting technical bulletins drafting. However, both GPT versions were criticized for producing overly generic advices, highlighting the importance of domain-specific training to integrate information on best management practices, locally authorized substances, and historical treatment schedules. Such implementation is crucial to align large language models with Integrated Pest Management principles and improve their precision towards their operational use.