Text Swin Transformer: a new transformer model for enterprise management text classification

As the Internet and information technology advance at a swift pace, enterprises have amassed vast quantities of data stemming from their operational activities, managerial processes, and other business functions. Behind these data lies extremely valuable knowledge that is closely related to the enterprise, which can provide critical decision-making support for enterprise management. Therefore, how to discover valuable knowledge from voluminous textual data, transform these textual data into a genuine asset for enterprises, and leverage this asset to inform decision-making and propel their growth, has become a challenging task. Inspired by this, this paper revises the Swin Transformer model, which has achieved wide success in computer vision tasks, and proposes a Text Swin Transformer (TST) model for enterprise management text data classification tasks. Specifically, we revise the original window attention, shifted window attention and patch merging strategies to text window attention, text shift window attention and window scaling strategies, in order to meet the needs of text classification tasks. In contrast to the state-of-the-art approaches in text classification task, the proposed model not only achieves excellent results on multiple standard datasets, but also requires less computation and boasts a faster computation speed.