Stealthy Query-Efficient OpaqueAttack Against Interpretable Deep Learning

Deep neural network (DNN) models are susceptible to adversarial samples in white-box and opaque environments. Although previous studies have shown high attack success rates, coupling DNN models with interpretation models could offer a sense of security when a human expert is involved. However, in white-box environments, interpretable deep learning systems (IDLSes) have been shown to be vulnerable to malicious manipulations. As access to the components of IDLSes is limited in opaque settings, it becomes more challenging for the adversary to fool the system. In this work, we propose a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Qu</i>ery-efficient <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Score</i>-based opaque attack against IDLSes, which requires no knowledge of the target model and its coupled interpretation model. By continuously refining the adversarial samples created based on feedback scores from the IDLS, our approach effectively reduces the number of model queries and navigates the search space to identify perturbations that can fool the system. We evaluate the attack's effectiveness on four convolutional neural network (CNN) models and two interpretation models, using both ImageNet and CIFAR datasets. Our results show that the proposed approach is query-efficient with a high attack success rate that can reach more than 95%, and an average transferability success rate of 69%. We have also demonstrated that our attack is resilient against various preprocessing defense techniques.