Consistency in causal reasoning for large language models in scenarios of HIV antiretroviral treatment, drug interactions, and side effects

We aimed at evaluating the consistency in causal reasoning capabilities of large language models (LLMs) in disease-specific contexts, across the three levels of the causal ladder-associational, interventional, and counterfactual. We aimed at obtaining realistic clinical scenarios involving drug interactions and side effects for Human Immunodeficiency Virus (HIV) antiretroviral therapy (ART). LLMs were prompted with context-rich clinical and causal vignettes to generate use cases, followed by context-simple queries and context-enhanced ones with the provision of extra documentation (either accessible or inaccessible), to test hallucinations. Multiple open- and closed-source LLMs, including GPT-4o and LLaMA-3, were evaluated for intra- and inter-model agreement (Fleiss' kappa), predictive performance with respect to generated scenarios (e.g., sensitivity, specificity), reasoning quality, alignment with clinical pharmacoepidemiology experts (Likert rates, Kendall's W), stratified by causal level. LLMs exhibited moderate to high intra-model agreement, decreasing from associational to counterfactual queries, whilst inter-model agreement was notably lower. References to inaccessible online content were produced. Experts revealed modest alignment with LLM outputs, diminishing confidence in the ability of LLM to generate reliable scenarios and ground truth. Associational reasoning was rated most favorably, whilst counterfactual scenarios demonstrated key weaknesses in clinical realism and depth. In conclusion, while generally concordant, LLMs are not fully competent in causal reasoning within HIV clinical pharmacoepidemiology.