Predicting the Impact of Scientific Articles Based on Citations

Evaluating the quality of the publication can depend on several factors, such as the importance of the contribution of the publication, the analysis of the results, and the number of citations, which is not feasible before publishing the article. In this paper, we fine-tuned the BERT and Longformer models to classify scientific articles into citation groups based on their significance using the article metadata, author affiliations, Web of Science journal categories, and the article full text. We collected 4234 open-access article metadata that were published in 2021 in Web of Science journals. Moreover, the full text of the article was also recovered from the article publishers. We fine-tuned the models on these 4234 articles, and the number of citations was taken in 2023, which the articles complete two years of publishing. We evaluated the models using precision, recall, and the F score, and Longformer slightly outperformed Bert, which scored 0.63 for the three metrics.