On contradictions between model interpretability and inversion attacks

For promoting the interpretability of Artificial Intelligence (AI) models, the methods of feature visualization, including Activation Maximization, help people understand how intractable Deep Neural Networks work by visualizing the representations of specific neurons. Meanwhile, the incredible-sounding ideas of reconstructing the input of an AI model through its output, or reconstructing the training data through simple access to the model, are indeed feasible. In fact, in order to explore the privacy leakage in AI models, model inversion techniques intend to reconstruct the private data through black-box or white-box access to AI models. Feature visualization and model inversion share a very similar framework and, in our point of view, this framework has great potential to be exploited for both beneficial and harmful intentions. In this paper, we uniformly refer to such operations of reconstructing data reversely as feature inversion. We will demonstrate feature inversion through a comprehensive analysis of model inversion and feature visualization, which are usually contradictory for the model trainer, as feature visualization boosts the interpretability of AI models while model inversion threatens privacy.