MACC: Masked Adversarial Convex Combination Against Word-Level Adversarial Attacks

Robustness againstword substitution attacks iscrucial for text classifiers and fundamental to broader NLP robustness. Such attacks use semantically similar word substitutions. Existing certified defenses often compute loose outer bounds for the convex hull of word embeddings, including irrelevant words, degrading performance on clean and adversarial tests. Additionally, convex hull-based methods also struggle to emulate worst-case scenarios accurately. This paper proposes the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Masked Adversarial Convex Combination</i> (MACC) method, which models the solution space as a convex hull of word vectors and uses Variational Information Bottleneck theory to mask unnecessary words. We also introduce an empirical masking method based on the volume of the convex hull to enhance the performance. By reducing the number of words explored within the convex hull, MACC enables more precise mimicry of worst-case attacks. Experiments across models and datasets show MACC outperforms existing methods in clean and adversarial accuracy against word-level attacks.