Novel approach of encrypted network traffic classification using deep convolutional neural network with Artificial Bee Colony and Genetic Algorithm

The encode network traffic makes it difficult to perform successful and dynamic classification. This paper will present the use of a hybrid model to be used with the publicly available QUIC dataset to classify VPN and non-VPN encrypted traffic based on a Deep Convolutional Neural Network (DCNN) and Long Short-Term Memory (LSTM) network, which is optimized by the Artificial Bee Colony (ABC) and Genetic Algorithm (GA). The method involves multi-angle processing - preprocessing, Min-Max normalization, and features selection of with correlation analysis, Fisher Score, and mutual information to obtain a tiny, but meaningful feature set (Size, Batch Cache, Delta Previous Packet). The chosen features are translated to 2D tensors through a sliding time window of consecutive packets, which allows the spatio-temporal DCNN+LSTM architecture to represent the level of intra- and inter-packet feature associations as well as inter- and intra-packet time dynamics. The disadvantages of single-optimization are overcome using a dual metaheuristic optimization strategy again whereby the work of the global hyperparameter exploration is done using ABC and the structural optimization is done using GA. The imbalance of classes is reduced with weighted loss functions and stratified data division. The accuracy of the model is 99.66% with 0.994 ROC-AUC and 0.987 PR-AUC and its MCC is 0.963 which is even greater than that of the traditional classifiers (Decision Tree, Random Forest, SVM, KNN), individual deep-learning models (CNN, LSTM), and image-based FlowPic method. Three-quarters of stratified cross-validation marks the case of consistent generalization (99.53% ± 0.09% mean accuracy), and an ablation study confirms the value of any one of the components. The findings prove that the presented framework can be applied to monitor the network traffic on encrypted networks which are security-sensitive and in real-time.