A review on explainable artificial intelligence in radiomics: State-of-the-art tools, prospective use cases, challenges and future directions

Radiomics converts medical images into high-dimensional quantitative features, yet the resulting predictive models often act as “black boxes,” limiting clinical adoption. Explainable AI (XAI) techniques offer a pathway to transparent, trustworthy decision support in radiomic workflows. We systematically survey leading XAI toolkits while evaluating their explanation paradigms, integration pathways, and performance characteristics. Building on these insights, we designed the XAIRadiomics architecture, which unites a data-fusion layer, a heterogeneous model suite (white-box, black-box, and hybrid learners), and a unified explainability module with intrinsic and post-hoc methods. Finally, we mapped key radiomic tasks (e.g., tumor detection, segmentation validation, outcome prediction) to XAI techniques and conducted a structured gap analysis. Our framework demonstrates how global attribution methods (i.e. cohort-level feature rankings) and local explainers (e.g., saliency maps, counterfactuals) can coexist within end-to-end radiomic pipelines. We illustrate use cases across oncology, neurology, cardiovascular imaging, multi-modal integration, pediatric applications, rare diseases, automated quality control, and surgical navigation. The key challenges highlight major hurdles—data heterogeneity, computational scalability, lack of standardized evaluation metrics, regulatory and ethical requirements, and end-user interpretation barriers along with the prospective future directions. Embedding XAI throughout radiomic workflows enhances model transparency, fosters clinical trust, and paves the way for personalized decision support. Future work should focus on hybrid explainability strategies, interactive/adaptive interfaces, benchmarking standards, seamless electronic health record (EHR) integration, robust ethical and regulatory frameworks, privacy-preserving federated XAI, and explicit uncertainty quantification. • Survey of XAI tools for radiomics, comparing strengths and limitations. • XAIRadiomics: modular end-to-end framework with preprocessing, fusion, and XAI. • Use cases include oncology, neurology, cardiology, pediatrics, QC, and surgery. • Challenges: protocol heterogeneity, scalability, benchmarks, and clinical usability. • Future work: hybrid XAI, FHIR/EHR integration, federated privacy, uncertainty.