Scale, Structure, and Stability: When Does LLM-Based Data Augmentation Improve Temporal Robustness in Web Intrusion Detection?

We investigate when LLM-based data augmentation can mitigate temporal collapse in web intrusion detection under extreme cross-temporal distribution shift. Under a strict hold-out protocol—training on CSIC-2010 and evaluating exclusively on the temporally separated SRBH-2020 golden set—the legacy-trained baseline exhibits near-collapse in balanced correlation (MCC ≈ 0) despite retaining high recall, revealing severe false-positive bias under drift. Rather than assuming uniform benefits from synthetic data, we analyze how augmentation effects vary with model scale. Using a fixed DeBERTa-v3-base backbone and five random seeds, we compare synthetic training corpora generated by multiple LLMs under identical schema-guided structured decoding and filtering constraints. The results reveal a scale-dependent threshold effect. Models below 12B parameters (i.e., the 4–8B settings in our experiments) frequently introduce structural artifacts that amplify false-positive bias and further destabilize cross-temporal performance. In contrast, models at or above 12B parameters consistently produce modest but statistically reliable recovery from correlation collapse (p < 0.001), with balanced metrics shifting toward the target-domain distribution. Although the absolute performance remains limited under extreme temporal separation, a confusion-matrix analysis shows that large-scale generation reduces false-positive skew and moves decision-boundary behavior closer to the modern-domain regime. These findings indicate that LLM-based augmentation is not inherently robustness-enhancing; rather, its effect depends critically on model scale and disciplined generation control. When properly constrained, ≥12B-scale models can partially stabilize cross-temporal behavior, whereas smaller-scale generation may exacerbate distributional fragility.