Mitigating ML-Driven Adversarial Attacks on xApps Using Dynamic Defense Mechanisms

The open radio access network architecture (O-RAN) architecture leverages intelligent near-real-time applications, known as xApps, to optimize network performance and services. However, these machine learning (ML)-driven xApps are vulnerable to adversarial attacks that can compromise their functionality and reliability. In this paper, we present a comprehensive study of adversarial threats targeting xApps and explore dynamic defense mechanisms to mitigate these risks. We begin by identifying potential attack vectors that target the near-real-time RAN intelligent controller (Near-RT RIC) and its associated xApps. We then utilize an open-source O-RAN testbed to deploy a key performance indicator (KPI) Monitoring xApp, a detection xApp, and a malicious adversarial xApp. The adversarial xApp carries out sophisticated inference-time attacks, including Carlini & Wagner (C&W) and basic iterative method (BIM), by perturbing key performance metrics in real-time to mislead the ML-based detection xApp. To counter these threats, we develop a defense xApp that integrates sequential anomaly detection techniques, ensemble deep neural network (DNN) inference, and gradient-based heuristics for real-time attack mitigation. Experimental results demonstrate that the C&W attack significantly degrades the baseline detection performance of the target xApp, reducing its accuracy from 92% to just 16% and BIM attack also achieves a comparable impact, lowering the detection accuracy to around 10%. Nevertheless, the proposed defense xApp promptly detects and neutralizes these adversarial manipulations, thereby restoring the effectiveness of the detector achieving up to 84% accuracy under the (C&W) attack and improving BIM (Basic Iterative Method) attack detection accuracy to 93% in the most challenging scenarios. This work presents a closed-loop evaluation of adversarial attacks and corresponding defenses within a real-world O-RAN environment that provides valuable insights into real-world vulnerabilities and mitigation strategies. By introducing a dynamic defense framework, we significantly enhance the security and resilience of ML-driven xApps and maintain reliable O-RAN performance even during attacks.