Seeing the Gaps: Heatmaps, Metrics, and the Hidden Failures of Explainable AI in Dentomaxillofacial Radiology – A Systematic Review

Abstract Objectives This systematic review aims to evaluate the techniques and reported patterns of heatmap-based visual explanations in deep learning (DL) systems for dental radiographic classification tasks. Methods A literature search following PRISMA guidelines was conducted in PubMed, Scopus, IEEE Xplore Library, Oria.no and Google Scholar to retrieve studies using DL for dental classification tasks. Only articles with a visual explanation (heatmap) were selected. Risk of bias assessment was done with QUADAS-2, PROBAST-AI and CEBM. The heatmap technique, reasons for inclusion, explanations provided and patterns were analysed. Results In the 136 studies reviewed, Grad-CAM was the most commonly used technique. The predominant reason for using visual explanations was to understand the decision-making process of the DL model. Some heatmaps reflect established radiographic criteria, while others exhibit unique, label-dependent patterns that sometimes emphasise irrelevant areas. Cropping to the region of interest influences visual explanations. Conclusions Visual explanations seem clinically plausible to domain experts when aligned with domain knowledge, but clinical plausibility does not equal technical faithfulness. Formal faithfulness testing was rarely performed among the included studies. Therefore, explanations should not be assumed to accurately represent the model’s underlying reasoning. Limitations include the evaluation of studies not primarily designed for visual explanation assessment. No funding was received for this review. Authors declare no competing interests. Advances in knowledge Visual explanations, while clinically plausible when aligned with domain knowledge, remain insufficiently validated and should not be assumed to faithfully represent model reasoning. An implausible explanation does not necessarily indicate model failure. Domain/task-specific evaluation metrics for explainable AI in dentomaxillofacial radiology are needed.