Bridging human-AI collaboration in healthcare: a systematic review of explainable AI applications

Explainable Artificial Intelligence (XAI) is increasingly being recognised as a key enabler of trust, transparency, and accountability in Clinical Decision Support Systems (CDSS). This research performs a systematic literature review of 225 peer-reviewed articles from 2015 to 2025 to discuss the current status of XAI in healthcare, outline dominant research trends, and shed light on emerging challenges. The results show a tenfold rise in global research output, driven by the need for explainable models to enhance diagnostic performance, patient safety, and clinician trust. The thematic clustering results in six main themes, including technical frameworks, human-centred design, ethical aspects, and regulatory compliance. Besides the technical contributions, this review underlines how interdisciplinary findings from the social sciences, ethics, and information science can provide the basis for defining responsible, sustainable use of XAI in the healthcare industry. This study concludes by providing practical recommendations and a research agenda that allow for human-AI collaboration, fairness, and transparency with consistency in policies.