Decentralized Federated Anomaly Detection in Mobile Networks

Networks face growing challenges such as Distributed Denial-of-Service attacks, malware propagation, unauthorized access, and irregular traffic patterns. Traditional centralized or static anomaly detection methods often struggle with scalability, latency, and privacy concerns, making them less effective in dynamic mobile environments. This research introduces a decentralized federated learning algorithm that integrates Multi-Agent Reinforcement Learning (MARL) with tree-based classification models, using XGBoost (Extreme Gradient Boosting) in the experimental setup to enhance anomaly detection while preserving privacy. MARL coordinates agent behavior to adapt to dynamic network conditions, promoting decentralized collaboration and resilience without raw data sharing. Each agent trains a lightweight tree-based model locally for efficient and scalable detection. Feature selection is performed collaboratively using reinforcement signals, while trust-weighted aggregation enables decentralized global learning without a central server. The system is designed to address key constraints such as intermittent communication, which is modeled through agent dropout in the experiments, as well as computational overhead and scalability in edge network environments. Experimental results using real-world intrusion detection datasets show that the proposed method achieves up to 93.5% accuracy, 86.1% precision and 85.7% F1 score, outperforming baseline methods by as much as 23.3% in F1 score under non-IID scenarios while maintaining low bandwidth and energy usage across dynamic agent participation and varying attacker models.