Assessing bias in the use of large language models in healthcare: a systematic review

Abstract The rapid advancement of Large Language Models (LLMs) has driven their growing adoption across domains, including healthcare. However, these models risk amplifying existing healthcare disparities. This systematic review synthesizes the existing literature on bias assessment in LLMs applied within the healthcare domain. We searched six databases, namely, Scopus, PubMed, Embase, Web of Science, IEEE Xplore, and the ACM Digital Library, for studies published between January 2020 and May 2025, with the final search conducted on 16 May 2025. Retrieved records were screened according to predefined criteria, resulting in 74 included articles. Methodological quality was assessed using the Joanna Briggs Institute Critical Appraisal Checklist for Analytical Cross-Sectional Studies. Owing to methodological heterogeneity, findings were described narratively. LLMs were applied across various healthcare contexts, most notably in medical question answering. Bias assessment methods ranged from human evaluations to statistical analyses, while only 13.5% of the included studies implemented mitigation strategies, which were primarily prompt-based. Overall, 87.8% of the studies reported evidence of bias, with demographic bias being the most commonly investigated bias type. The included studies were of predominantly high quality with minimal risk of bias. The findings indicate that bias remains prevalent in LLM healthcare outputs, highlighting a critical challenge to their equitable and responsible deployment. Future studies should prioritize robust bias evaluation and the implementation of effective bias mitigation strategies to support the safe integration of LLMs into healthcare.