A Dual-Approach Framework for Enhancing Network Traffic Analysis (DAFENTA): Leveraging NumeroLogic LLM Embeddings and Transformer Models for Intrusion Detection

In cybersecurity, network traffic analysis is essential for identifying abnormal patterns that may indicate cyberattacks, often beyond the capabilities of human detection. While Machine Learning (ML) has proven effective in this domain, traditional ML algorithms face significant challenges when dealing with large-scale data and class imbalances, which are commonly found in network traffic logs. These challenges can compromise model accuracy and reliability. To address them, we propose a novel Dual-Approach Framework for Enhancing Network Traffic Analysis (DAFENTA): integrating Large Language Model (LLM) embeddings with NumeroLogic encoding to enhance feature representation and employing a Transformer encoder model for direct binary classification of network traffic. By embedding network traffic data using the sentence-transformers model, we improved feature contextualization for typical ML classifiers like Random Forest, AdaBoost, Gradient Boost, Extra Trees, Logistic Regression, and KNN. In addition, the numerical reasoning is enhanced in LLM by applying the NumeroLogic encoding. At the same time, we leveraged Transformer models to capture complex feature dependencies in data through attention mechanisms. This framework was evaluated through the KDD and ISCX traffic flow datasets. Our results have shown that LLM embeddings, when integrated with NumeroLogic encoding method, significantly enhanced the performance of traditional ML models by improving feature generalization, especially in handling out-of-distribution samples. Additionally, the Transformer model demonstrated efficiency in managing large-scale data and addressing class imbalance, leading to marked improvements in accuracy. This dual-approach framework offers a powerful method for advancing anomaly detection in network traffic analysis, leveraging the capabilities of Large Language Models.