Mitigating medical bias in large language models by prompt engineering: an empirical study of effectiveness and trade-offs

Large language models (LLMs) demonstrate expert-level performance in various medical scenarios, yet their outputs can exhibit bias against groups or individuals with specific sensitive attributes, posing risks to patient safety and undermining trust in LLMs for healthcare. Recent research suggests that prompt engineering offers a convenient way to adjust model outputs, with the potential to mitigate such biases. However, there is a lack of empirical studies that systematically examine the effectiveness of prompt engineering and its trade-offs among fairness, accuracy and inference overhead. To fill this gap, we empirically evaluate five widely used prompting strategies across five influential LLMs in the latest medical bias benchmark. Results reveal substantial heterogeneity in both effectiveness and overhead across models, with no strategy proving universally effective and some even exacerbating bias. Chain-of-thought prompting yields the largest reduction, lowering the average gap across all scenarios by 2.4 percentage points, where the largest reduction is 6.2 percentage points, obtained in the DeepSeek-V3.1-sex case. Furthermore, the results of the McNemar test also show that it achieves the largest number of significant bias reduction cases (8/15), primarily by improving performance on unprivileged groups. These findings provide practical guidance for the fair deployment of LLMs in healthcare and highlight that mitigating medical bias remains a challenging problem requiring sustained efforts from both the artificial intelligence (AI) and medical communities. To support future research on fair AI in healthcare, we shall release all results and source code. This article is part of the theme issue 'Safe, secure and robust AI for safety-critical systems'.