Long-text classification in Chinese via fasttext and multihead attention with bigram and trigram

In the realm of Chinese long-text categorization, the advent of the big data era has led to an exponential surge in information volume, thereby imposing stricter demands on the accuracy and efficiency of automatic classification systems. This study introduces an enhanced model, hereby referred to as "FastText-BT-Att" based on the FastText framework. This model further deepens text representation through the integration of Bigram and Trigram features and incorporates a Multi-Head Attention mechanism to finely identify and weigh key information within the text. Comparative experiments demonstrate that the FastText-BT-Att model achieves higher accuracy rates in long-text classification tasks. The model is significantly outperforming traditional models, underscoring its enhanced effectiveness, highlighting its effectiveness and superiority in the task of categorizing Chinese long texts.