A Resource-efficient Text-to-Text Transfer Transformer Encoder-based Vertical Hybrid Model for Malicious URLs Detection

With its flexible text transformation capabilities and pre-training background, Text-to-Text Transfer Transformer (T5) is well-suited for malicious Uniform Resource Locators (URLs) detection. However, due to the complex structure, it requires longer training processes, more computational resources, and suffers from slow speed of decision-making. To solve these challenges, in this paper we propose a novel hybrid model, T5-BiGRU-FC, which integrates T5 encoder, bi-directional gated recurrent units (BiGRU), and a fully connected layer for the first time. We selected T5-small and T5-base as baselines, set different learning rates for training, recorded the training process data, and evaluated their performance on testing set. The results show that, compared to baselines, our proposed models reduced the average training time per epoch by more than 55%, and increased the average decision-making speed by more than 80%, with the maximum reaching 133.33%. Additionally, trained with a higher learning rate, the proposed T5-small-BiGRU-FC and T5-base-BiGRU-FC achieved accuracy of 92.3% and 95.2% respectively, with slight improvements in precision and recall. The comparison results fully validate the rationality and potential of our proposed hybrid model in malicious URLs detection tasks.