Img2Tab: Automatic Class Relevant Concept Discovery from StyleGAN Features for Explainable Image Classification

Traditional tabular classifiers provide explainable decision-making with interpretable features (concepts). However, using their explainability in vision tasks has been limited due to the pixel representation of images. In this paper, we design Img2Tabs that classify images by concepts to harness the explainability of tabular classifiers. Img2Tabs encode image pixels into tabular features by StyleGAN inversion. Since not all of the resulting features are class-relevant or interpretable due to their generative nature, the Img2Tab classifier should automatically discover class-relevant concepts from the StyleGAN features. Thus, we propose a novel algorithm using the Wasserstein-1 metric to quantify class-relevancy and interpretability simultaneously. By this method of concept visualization, we quantitatively investigate whether important features extracted by tabular classifiers are class-relevant concepts. Consequently, we determine the most effective classifier for Img2Tabs in terms of discovering class-relevant concepts automatically from StyleGAN features. In evaluations, we demonstrate concept-based explanations through importance and visualization. Img2Tab achieves top-1 accuracy on par with CNN classifiers and deep feature learning baselines. Additionally, we show that users can interactively debug Img2Tab classifier to prevent erroneous decision-making from data bias without sacrificing accuracy. The source and demo code for Img2Tab are available at https://github.com/songsnim/Img2Tab_pytorch