Hierarchical spatio-temporal encoding with prototype-conditioned GAN for network traffic anomaly detection

Current network traffic anomaly detection systems fail to reliably identify rare zero-day exploits and privilege-escalation attacks because the extreme scarcity of minority-class samples prevents models from learning coherent decision boundaries for these categories. To close this gap, we propose PRISM (Prototype-guided Representation with Imbalanced Spatio-Temporal Modeling), a framework that simultaneously addresses dynamic graph topology and severe class imbalance. PRISM employs three key components. First, a Dynamic Adaptive Graph Construction (DAGC) mechanism that learns a task-sensitive sparse adjacency matrix end-to-end. Second, a Hierarchical Spatio-Temporal Encoder (HSTE) that fuses graph topology and causal flow history via bidirectional gating. Third, a prototype-conditioned Wasserstein GAN with Gradient Penalty (WGAN-GP) that anchors minority-class synthesis to dynamically updated semantic prototypes, directly addressing class imbalance without modifying the inference pipeline. The framework adds negligible inference overhead, requires no architectural modifications to the downstream classifier, and is therefore suitable for deployment in high-speed network monitoring environments. We evaluate PRISM on the NSL-KDD, CICIDS2017, and UNSW-NB15 public benchmarks. PRISM achieves peak F1-scores of 99.21%, 98.87%, and 93.47% on these datasets, representing improvements of 0.42%, 0.31%, and 0.73% over the baseline BiNF model.