Multiscale Feature-Guided Adversarial Examples Quality Assessment via Hierarchical Perception of Human Visual System

Deep neural networks (DNNs) reveal significant robustness deficiencies due to their susceptibility to being misled by small and imperceptible adversarial examples, thus it is crucial to improve the robustness of DNNs against such harmful perturbations. The current <inline-formula><tex-math notation="LaTeX">$L_{p}$</tex-math></inline-formula> specification ignores differences in human visual perception when measuring similarity, and most existing image quality assessment (IQA) methods and adversarial example datasets lack subjective scores for evaluation. In this paper, we construct a new database of adversarial examples, called the AED, which contains 35 original images, 1050 adversarial examples, and the corresponding subjective scores of adversarial examples. Then, a novel full-reference IQA model for the quality evaluation of the adversarial examples is proposed by taking into full consideration the hierarchical perception of human visual system (HVS) and the outstanding capabilities of the multi-scale feature extraction network in feature extraction. Specifically, a feature encoding network that uses continuous convolution layers to pre-extract features and expand the receptive field of the image is employed. To simulate the HVS hierarchical perception, the features of different scales are further obtained by designing a multi-scale feature extraction network. The structural similarity scores of the feature maps at different scales are calculated for jointly arriving at the final IQA score of the adversarial examples. Experimental results have demonstrated that our proposed model is closer to the perception of HVS in small imperceptible distortions evaluation of adversarial examples compared with other classical and state-of-the-art models.