Study for text length impact on text classification accuracy based on the transformer method

One of the key tasks in natural language processing is text classification, while traditional text classification methods have limitations in processing long texts and complex dependencies. The Transformer model can effectively capture local and global dependencies in texts to better understand the meaning of texts and can process texts of arbitrary length. This study explores the performance of the Transformer-based text classification model under different text lengths. A component of the research methodology is utilizing the THU Chinese Text Classification (THUCNews) dataset, which comprises around 740,000 news articles categorized into 14 groups. Data preprocessing includes word segmentation, stop word removal, and stemming. Evaluation measures such as recall, accuracy, precision, and F1 score are used to fully assess the model's performance The Transformer-based text categorization model exhibits notable variations depending on the text length, according to experimental data. Specifically, long texts with large information capacity can usually provide more contextual information, which helps to improve classification accuracy. However, when the text length approaches an extreme, the accuracy may not follow this trend, possibly because the number of samples is too small or contains noisy data. The results analysis and discussion highlight the need for an adequate and uniformly distributed number of text samples varying in length to enhance the validity and generalizability of the experimental findings. In addition, special processing of texts of extreme lengths, such as data enhancement or feature extraction, can increase the amount of training data for the model.