Generalizability of Concept Knowledge in Machine Learning Using TCAV Scores: A Case Study Using Different Skin-Lesion Datasets

In safety-critical fields, such as skin-lesion classification, interpretability of the decisions of a machine learning model is required. This can be provided through concept-based interpretability methods like testing with concept activation vectors (TCAV). TCAV quantifies how specific human-understandable concepts influence a model’s decisions. A further issue affecting the performance of ML models is generalizability, i.e. , how well a model generalizes to unseen data from a different domain. It is currently unknown how the interpretability provided by TCAV is affected by domain shifts. Here we show that TCAV-based interpretability is predominantly unaffected by domain shifts. To that end, we introduce concept detection scores (CDS) as aggregated TCAV scores which are directionally unified and thus a suitable evaluation metric. The results show only small differences between CDS within domain and across domain for 48 models trained on three distinct source domains. This increases the viability of TCAV as an interpretability tool since it can be used without additional effort to manage generalizability.