GRVT: Improving the Transferability of Adversarial Attacks Through Gradient Related Variance and Input Transformation

ABSTRACT As we all know, the emergence of a large number of adversarial samples reveals the vulnerability of deep neural networks. Attackers seriously affect the performance of models by adding imperceptible perturbations. Although adversarial samples have a high transferability success rate in white‐box models, they are less effective in black‐box models. To address this problem, this paper proposes a new transferability attack strategy, Gradient Related Variance and Input Transformation Attack (GRVT). First, the image is divided into small blocks, and random transformations are applied to each block to generate diversified images; then, in the gradient update process, the gradient of the neighbourhood area is introduced, and the current gradient is associated with the neighbourhood average gradient through Cosine Similarity. The current gradient direction is adjusted using the associated gradient combined with the previous gradient variance, and a step size reducer adjusts the gradient step size. Experiments on the ILSVRC 2012 dataset show that the transferability success rate of adversarial samples between convolutional neural network (CNN) and vision transformer (ViT) models is higher than that of currently advanced methods. Additionally, the adversarial samples generated on the ensemble model are practical against nine defence strategies. GRVT shows excellent transferability and broad applicability.