Causal networks guiding large language models: application to COVID-19

In the context of diagnosis of COVID-19, this paper shows how to convert a Causal Network to a Large Language Model (LLM). The Causal Network was converted to the language model using prompts and completions. Prompts were composed from the full-factorial combination of the text associated with statistically significant variables in the Causal Network. Completions were based on the evaluation of the probability of COVID-19 using the Causal Network. The accuracy of the Causal Network and LLM was tested using two databases. The first database was based on a survey of 822 patients, collecting 12 direct (parents on the Markov blanket of COVID-19 diagnosis node), 7 indirect (associated with COVID-19 but not direct cause) symptoms of COVID-19. The second set was based on 80 patients reporting their symptoms in open-ended questions, often reporting some of the direct predictors and rarely reporting any indirect predictors of COVID-19. The accuracy of Causal Network and Markov blanket was tested using Area under the Receiver Operating Curve (AUROC). When indirect information was available, the Causal Network model (AUROC = 0.91) was significantly more accurate than the LLM (AUROC = 0.88), even though LLM model was trained to duplicate predictions of the Causal Network. Where the indirect information was not available, both models had lower accuracy (AUROC of 0.75 and 0.76). The accuracy of LLM depends not only on patterns among direct predictors of the outcome but also data not reported to the LLM. Conversational LLMs need to go beyond information the patient supplies and proactively ask about missing, typically indirect, information.