Deep Learning Intrusion Detection: A Critical Reflection on the Inherent Biases of Datasets and Evaluation Misconceptions

Deep learning-based intrusion detection is an important means of network security protection, but the existing validation and evaluation mechanisms still have many systematic problems. This paper discusses three aspects: mainstream validation methods, inherent biases in public datasets, and the failure of evaluation metrics, and explains the reasons for the “false prosperity” in this field. Models cannot truly understand attack characteristics and only remember the statistical features of old datasets. When the sample classes are imbalanced, metrics such as accuracy and F1 can easily be misleading, and the good performance of models on test sets is difficult to reflect their actual detection effect. This paper points out that slow data updating, unreasonable evaluation standards, and insufficient model robustness are the main difficulties at present, and puts forward some ideas for establishing objective evaluation standards. This review hopes to remind researchers to pay attention to the reliability of model validation and promote deep learning-based intrusion detection from laboratory testing to real network protection scenarios.