Multi-label Topic Classification of Research Papers Using Transformer-based Models

Purpose To develop MLP and transformer-based models for the multi-label topic classification of research papers using abstract text.Methods Abstracts from 119,600 papers in the Computer Science category of arXiv were collected to create a multi-label dataset with up to three categories out of a total of 15 possible categories. Performance was evaluated by developing a baseline MLP model along with transformer-based models: BERT, RoBERTa, and DistillBERT.Results The transformer models outperformed the traditional MLP model. The DistillBERT model achieved the highest micro F1-score of 0.749, while the BERT model recorded macro and weighted F1-scores of 0.655 and 0.733, respectively. The RoBERTa model excelled in the samples method with a score of 0.772.Conclusion This study enables researchers to quickly explore recent findings and effectively identify their research topics. Additionally, it is expected to significantly contribute to the efficient sharing of academic knowledge and the revitalization of the research community.