Using iterative agentic retrieval-augmented generation to refine the risk of infection from social contexts during the COVID-19 pandemic: Development and Evaluation Study

Background: Contact tracing is crucial to limit pathogen transmission and to gather epidemiological data. However, despite machine learning optimizations, it remains time-consuming. Large language models have been used for infection risk prediction, but no existing application converses with a user to adapt its risk estimates. Objective: We aimed to develop an iterative retrieval-augmented generation conversational framework to predict coronavirus disease 2019 (COVID-19) infection risk from textual social data. The conversational agent aims to gather personalized risk factors based on descriptions of social interactions. Methods: We trained a long short-term memory (LSTM) model to predict the risk of COVID-19 infection from 19,434 textual descriptions of social interactions, collected during the pandemic in the Leuven student community, in Belgium. We integrated this model into a Llama3.3-based conversational framework that iteratively gathers additional social information that may represent a risk of infection. We evaluated the framework using large language model (LLM)-as-a-judge metrics and by comparing its risk assessment in different social contexts. Results: Based solely on social interaction data, a fine-tuned RobBERT model predicted infection risk with an area under the receiver operating characteristic curve (ROC AUC) of 0.73. It performed similarly to an LSTM model and a logistic regression on term frequency-inverse document frequency (TF-IDF) features, which both achieved a ROC AUC of 0.72. These three models slightly outperformed a Quantized Low-Rank Adaptation (QLoRA)-fine-tuned Mistral NeMo model (ROC AUC = 0.69). Llama3.3 performed well in our framework: the predicted infection risk converged often, and it asked fewer redundant questions than the smaller models (Mistral 7B, Gemma3 12B). In addition, the framework showed the ability to stratify, by risk of infection, synthetic users who had different types of social interactions. Conclusions: Combining simple language models with large language models can refine individual infection risk from contact tracing data in a high-circulation context. Our conversational framework may help optimize contact tracing by highlighting social contexts with a high risk of infection.