Enhancing Public Health: training an AI chatbot for complex tasks support

Abstract Background The evolution of Large Language Models (LLMs) such as GPT-4, Gemini, and Copilot has transformed the field of Natural Language Processing, demonstrating human-level capabilities and potential applications in Public Health. Here, we report on the training of a chatbot model based on GPT-4 designed to assist operators with relevant and complex public health tasks. Methods A scoping review of existing studies on the application of LLMs in Public Health settings was conducted to understand the landscape and identify gaps in their current implementations. We then proceeded to analyse the necessary skills and competencies required for public health professionals to adequately instruct the model on required tasks. Finally, we selected the Italian regulations and guidelines with the greatest relevance in the public health field to provide the knowledge base to the model. Results The review revealed no relevant documents on the use of LLMs in areas related to Public Health. Skills and competencies were retrieved from the Guidelines of the Italian Society of Hygiene for Public Health operators and used to instruct the model on how to address public health tasks. As for the knowledge base, we retrieved 25 documents regulating specific areas of the public health field, such as vaccinations, hygiene and safety at work, infection prevention, and school hygiene. Such documents were used to train the model, after formatting them appropriately. Conclusions Despite challenges with non-standardized and technically complex documentation, preliminary testing of our model has shown its versatility and utility in supporting daily public health operations. A validation phase is under definition and will involve a DELPHI process, in which products of the custom GPT model will be evaluated by a panel of public health operators to evaluate its performance in real-world settings, ensuring compliance with national health regulations. Key messages • Training a GPT-4-based chatbot to support complex tasks in Public Health, tailored by relevant national guidelines. • Our custom model, informed by public health guidelines, showed potential in preliminary tests.