Density-aware and Cluster-based Federated Anomaly Detection on Data Streams

Federated active anomaly detection on data streams becomes a crucial research problem, since it attempts to discover anomalous data with protecting data privacy and avoiding extensive data labeling. Although extensive work has been conducted on anomaly detection, distinguishing similar anomalies of different categories still remains quite a challenging issue. The requirement of privacy protection in federated settings aggravates the difficulties for instance query and scoring in active anomaly detection when solving this issue. To the best of our knowledge, limited work has focused on this research area. Therefore, we propose Density-aware and cluster-based Federated Active anomaly detection on data Streams, called DFAS. We design a novel lightweight federated anomaly detection clusters with density-aware hash cells, which successfully capture evolving data distribution. The federated anomaly detection clusters are incrementally updated with an acceptable theoretical reconstruction error guarantee. In addition, we propose a straightforward but effective metric divergences accompanied by a greedy search algorithm, which takes both global aggregation bias mitigation and efficiency into account. At last, DFAS detects anomalies and queries the instances for manual labels by measuring the density in hash cells of each cluster, effectively distinguishing closely distributed anomaly classes while maintaining data privacy in the federated setting. Comprehensive experiments on several real-world data sets show that DFAS outperforms previous methods, improving F1 scores by up to 26.7%.