Adversarial Robustness of Semantic Cache Verifiers: Gaps Exposed by Red-Teaming and Partial Repair via Training

The main paper (Part I of this series) shows that fine-tuning a lightweight cross-encoder verifier on in-domain gray-zone labels reliably repairs its discriminative gap on natural benchmark data. But natural benchmark data contains only the hard cases a dataset happens to include, not cases deliberately designed to fool the verifier. This paper measures that worst case systematically, using an LLM to generate adversarial samples covering five known failure axes (negation, action-verb swap, direction reversal, named-entity swap, and quantity/parameter swap). Across 306 adversarial samples that clear the gray-zone similarity threshold, an off-the-shelf verifier's false-accept rate reaches 84.0% (95% CI [79.7%, 87.9%]), far beyond anything measured on natural data. More importantly, the in-domain fine-tuning the main paper validates as an effective remedy provides no protection at all on these adversarial samples (fine-tuned false-accept rate 87.6%, its confidence interval almost fully overlapping the untuned one) - fine-tuning learns what a dataset's natural noise looks like, not general robustness to deliberate attacks. A direct remedy was tested: mixing adversarial samples targeted at these five failure axes, only 3.8% of the training set, into the original natural training data and re-fine-tuning. This drops the false-accept rate to 53.6% (95% CI [48.0%, 59.2%]) without sacrificing natural-data discriminative power (AUC rises from 0.7212 to 0.8749). This suggests the adversarial-robustness gap is not a fundamental limitation of the fine-tuning mechanism itself, but 53.6% still means more than half of adversarial samples fool the verifier - the problem is substantially mitigated, far from solved. This is Part II of a four-part series distilled from a single complete technical report (Zenodo DOI: 10.5281/zenodo.21703364), and a direct companion to Part I (Xin, C. (2026). Zenodo. https://doi.org/10.5281/zenodo.22660442).