Using Shapley Values to Explain the Decisions of Convolutional Neural Networks in Glaucoma Diagnosis

This work aims to leverage Shapley values to explain the decisions of convolutional neural networks trained to predict glaucoma. Although Shapley values offer a mathematically sound approach rooted in game theory, they require evaluating all possible combinations of features, which can be computationally intensive. To address this challenge, we introduce a novel strategy that discretizes the input by dividing the image into standard regions or sectors of interest, significantly reducing the number of features while maintaining clinical relevance. Moreover, applying Shapley values in a machine learning context necessitates the ability to selectively exclude features to evaluate their combinations. To achieve this, we propose a method involving the occlusion of specific sectors and re-training only the non-convolutional portion of the models. Despite achieving strong predictive performance, our findings reveal limited alignment with medical expectations, particularly the unexpected dominance of the background sector in the model’s decision-making process. This highlights potential concerns regarding the interpretability of convolutional neural network-based glaucoma diagnostics.