Paving the Roadmap for XAI and IML in Healthcare: Data-Driven Discoveries and the FIXAIH Framework

Artificial intelligence is revolutionizing healthcare. However, its adoption is hindered by the technical-proficiency gap, the disconnect between AI complexity and clinical interpretability. Although Explainable AI (XAI) and Interpretable Machine Learning (IML) techniques aim to address this gap, fragmented research limits the development of generalizable and practical guidance. In response, we introduce FIXAIH, a dynamic, evidence-based framework that consolidates fragmented research in XAI and IML into a coherent structure to guide implementation in healthcare. It is designed to enhance the transparency, accountability, and efficacy of AI systems -- characteristics essential to meet trust and legal requirements, e.g., in the EU AI Act. FIXAIH uses a machine learning-based methodology to holistically analyze literature to identify key parameters shaping the integration of XAI and IML in clinical contexts. We develop a BERT-based tool and examine 5,083 articles, identifying 13 key parameters across three macro-parameters: Research Methods, Health Disorders, and Disease Prevention. This categorization, refined through review of 200+ studies, clarifies applications, trends, and challenges in healthcare XAI. These parameters serve as design elements for developing and aligning AI systems. Drawing on them, FIXAIH supports diagnostic accuracy, treatment efficacy, and risk prediction through, e.g., feature relevance, bias detection, model refinement, data preparation, and expert-guided adjustments. FIXAIH embeds this methodology and adaptable guidelines at its core, evolving as new evidence becomes available, with the intention to expand through integration of social media, clinical insights, regulatory data, and other sources. This adaptive design supports continuous learning and practical implementation, promoting a trustworthy approach to medical innovation.