Network Intrusion Detection Using Transformer Models and Natural Language Processing for Enhanced Web Application Attack Detection

The increasing frequency and complexity of web application attacks necessitate more advanced detection methods. This research explores integrating Transformer models and Natural Language Processing (NLP) techniques to enhance network intrusion detection systems (NIDS). Traditional NIDS often rely on predefined signatures and rules, limiting their effectiveness against new attacks. By leveraging the Transformer's ability to capture long-term dependencies and the contextual richness of NLP, this study aims to develop a more adaptive and intelligent intrusion detection framework. Utilizing the CSIC 2010 dataset, comprehensive preprocessing steps such as tokenization, stemming, lemmatization, and normalization were applied. Techniques like Word2Vec, BERT, and TF-IDF were used for text representation, followed by the application of the Transformer architecture. Performance evaluation using accuracy, precision, recall, F1 score, and AUC demonstrated the superiority of the Transformer-NLP model over traditional machine learning methods. Statistical validation through Friedman and T-tests confirmed the model's robustness and practical significance. Despite promising results, limitations include the dataset's scope, computational complexity, and the need for further research to generalize the model to other types of network attacks. This study indicates significant improvements in detecting complex web application attacks, reducing false positives, and enhancing overall security, making it a viable solution for addressing increasingly sophisticated cybersecurity threats