Unveiling Themes in Social Media Data: A Two-Stage Hashtag-Driven Clustering and Classification Method in Uzbek Language

In this article, the authors developed a two-stage method for analyzing user comments from social networks. At the first stage, the authors cluster comments by hashtags using K-Means, where they obtained 18 clusters at the output. From these classers, eight supercategories of comments were formed, which were used to train the classification model. The multilingual neural network model was used as a model, which was further trained on the formed corpus mentioned above. Experiments have shown that using hashtags as “weak” labels can significantly reduce the labor costs of manual labeling and at the same time achieve fairly high classification rates. On a separate test sample of 2000 comments, Precision and Recall reached average values of 0.81 and 0.78, respectively, with a final F1-measure of 0.79. The article also analyzes typical errors (false positive and false negative) related to the intersection of topics, polysemy of terms, and the specifics of the informal language of social networks. The proposed approach can be useful for automatic monitoring of public opinion, identifying trends, and solving other applied problems of content analysis.