CNNRes-DIndRNN: A New Method for Detecting TLS-Encrypted Malicious Traffic

While ensuring the accuracy of encrypted malicious traffic detection, improving model training speed remains a challenge. In order to solve this challenge, we propose CNNRes-DIndRNN for detecting encrypted malicious traffic classification. This model uses 1D-CNN to capture local feature relationships between data and IndRNN to capture their global dependency relationships. This method uses Zeek (version 7.0.0) to filter TLS datasets and NetTiSA to build time-series features that help models identify malicious behaviors. Combine time-series and encrypted features, then encode them with XLNet to improve model learning ability and speed training. In the final step, the encoded data is fed into CNNRes-DIndRNN. The results on five datasets including CTU-13 and MCFP showed that CNNRes-DIndRNN achieved 99.81% accuracy in binary classification and 99.67% in multi-class classification. These results represent improvements of 0.50–7.78% (binary) and 0.93–12.26% (multi-class) over all baseline methods. In performance comparisons, CNNRes-DIndRNN achieved the fastest training and testing times. It achieves the best comprehensive performance while maintaining high recognition accuracy.