Chinese Text Classification Based on Different Word Segmentation Methods

This study investigates the impact of different Chinese word segmentation methods on classifying classical literary works. A variety of datasets, including novels and biographies, were collected, and the Term Frequency-Inverse Document Frequency (TF-IDF) algorithm was employed for feature extraction. Character-based segmentation, treating each character as a token, was compared with multi-character word-based segmentation. U sing support vector machines (SVM) for classification, results indicated that multi-character phrase extraction outperformed character-based segmentation in accuracy for both large and small datasets. This highlights the significance of word segmentation techniques in enhancing text classification performance in Chinese literature. Future research could explore additional segmentation methods and their applications.