Boosting Adversarial Training With Mitigating Hard Sample Interference

Adversarial training (AT) has shown impressive advantages in maintaining accuracy and enhancing robustness against adversarial examples. However, most existing AT techniques jointly optimize clean-example accuracy and adversarial-example robustness as dual objectives. This setting introduces an often overlooked issue; when optimizing hard samples near the decision boundary, the model may bolster robustness at the expense of accuracy or preserve accuracy to the detriment of robustness. To alleviate the accuracy-robustness sacrifices induced by hard samples, we propose mitigating hard sample interference (MHSI) from a sample-intervention perspective. MHSI aims to reduce the instability caused by hard samples during AT. Specifically, we introduce a weighted adaptive (WA) mechanism that strengthens the model's learning of clean samples, thereby reducing the negative impact of hard samples on accuracy. In addition, guided by an analysis of the gradient norm and the Hessian matrix, we design a dynamic calibration (DC) strategy that dynamically calibrates the probability outputs of hard samples to mitigate their damage to robustness. With these two modules, our approach significantly improves robustness without sacrificing accuracy. Extensive experiments on CIFAR-10, CIFAR-100, Tiny ImageNet, and SVHN demonstrate that MHSI effectively improves both accuracy and robustness and outperforms state-of-the-art methods under glass-box attacks. Notably, under an $l_{\infty }$ attack, MHSI yields up to a 6.22% robustness gain over the AT baseline. Our code is available at https://github.com/hubin111/MHSI.