Autonomous Data Scientist (ADaS): A Practical Example of the Future Role of AI in Cybersecurity

The volume of network traffic and elusiveness of modern cyber threats challenge the ability of cyber analysts to identify anomalous behavior in networks. In this work, we propose an autonomous agent to enhance analysts’ detection capabilities. To this end, we designed an agent to analyze feature-rich network traffic by autonomously executing the Data Analytic Development Process: (1) data engineering–prepares datasets, (2) feature engineering–selects the most representative features, (3) model engineering–chooses the best algorithm-feature combination, and (4) analytic deployment–applies the optimized analytic to the dataset and identifies anomalous behavior. We refer to this agent as an Autonomous Data Scientist (ADaS), utilizing reinforcement learning as an orchestrator to determine the optimal combination of features and unsupervised clustering algorithm. ADaS operates without labeled training data, and results on popular public datasets (NB15, IoT-23, and KDD’99) demonstrate high detection and low false alarm rates, making it reliable and novel for anomaly detection.