GTAE-IDS: Graph Transformer-Based Autoencoder Framework for Real-Time Network Intrusion Detection

Network intrusion detection systems (NIDS) utilize signature and anomaly-based methods to detect malicious activities within networks. Advances in machine learning (ML) and deep learning (DL) algorithms have enabled NIDS to analyze large volumes of data and identify complex patterns. However, traditional ML/DL approaches in NIDS have primarily relied on flow-based features and utilized flat data formats, such as vectors or grids, which limit their ability to recognize the structural and contextual nuances of network attacks, particularly in real-time. Additionally, most NIDS depend on supervised or semi-supervised learning, requiring extensive labeled data that is timeconsuming to generate and not always feasible. This reliance restricts their ability to detect novel attacks, as they typically only recognize threats similar to those encountered during training. Hence, there is a significant need to develop NIDS that can operate in near real-time, eliminate the need for labeled data, and effectively identify novel attack patterns. We propose GTAE-IDS, a novel unsupervised packet-based graph neural network framework aimed at early and precise anomaly detection in network traffic. GTAE-IDS employs graph embeddings to capture and process network traffic data swiftly, creating sequential packet-based graphs that reflect network communications. Our approach employs graph autoencoders to identify structural and global patterns in benign data without needing labeled graph data, enhancing detection capabilities against novel attacks. Incorporating transformers in the encoder segment, GTAE-IDS effectively discerns contextual patterns in network traffic, achieving over 98% accuracy in identifying malicious activities on benchmark network intrusion data sets.