Reliable evaluation for the AI-enabled intrusion detection system from data perspective

As the primary link in cybersecurity, the intrusion detection system (IDS) is of indispensable importance. Many studies have proposed sophisticated artificial intelligence (AI) models to detect intrusion behavior from a large amount of data, yet they have ignored the fact that poor data quality has a direct impact on the performance of IDS. The poor data quality is mainly attributed to the interference and damage, such as data tampering, poisoning, and corruption, which leads to decision-making deviations, triggering a serious trust crisis of model application. This paper proposes a multi-indicator comprehensive evaluation method (MICEM) to ensure the reliability of AI decision-making from data perspective. First, several evaluation indicators are established to analyze the potential risks that intrusion detection data may face from the different dimensions, and specific quantitative methods are provided. Second, a comprehensive evaluation is conducted based on the results of each indicator to determine the quality of the intrusion detection data as a whole, thus guaranteeing the usability and reliability of AI-enabled IDS. Finally, the effectiveness and practicality of the proposed MICEM are fully verified by evaluating the benchmark-CICIDS2017 dataset and the real intrusion detection dataset.