Progressive Adversarial Contrastive Learning: Towards Efficient Data Augmentation in Adversarial Defense

In the realm of deep learning, adversarial attacks have surfaced as a significant concern, obstructing the deployment of deep neural networks (DNNs) in security-sensitive sectors. The cost of data annotation, increasing with data volume, has led to the exploration of adversarial training within self-supervised learning frameworks as a prevalent defense solution. Improving adversarial training is essentially a problem of exploring robust and efficient data augmentation. However, the computationally demanding nature of adversarial augmentation within a self-supervised learning framework poses dual challenges concerning performance and efficiency. In this study, we introduce Progressive Adversarial Contrastive Learning (PACL), an innovative defense mechanism against adversarial attacks on neural networks, which incorporates efficient adversarial data augmentation tactics. Specifically, PACL utilizes a sinusoidally enhanced adversarial sample strength growth strategy and has a universal adversarial strength control. This approach mitigates the inefficient computation of adversarial samples during the initial training phase, resulting in data augmentations that more closely mirror potential malicious inputs post-model deployment. Through exhaustive experiments against one supervised and four self-supervised competitive baselines, we empirically substantiate the effectiveness of PACL. Our results demonstrate a 33.90% reduction in computational overhead and an increase in robustness of up to 10.20%.