Document Classification Based on Computational Models: Insights from Computer Science Case Studies

This study investigates the effectiveness of various computational models in classifying scientific abstracts from arXiv. It compares traditional machine learning methods— Multinomial Naive Bayes, Logistic Regression, and Support Vector Machines (SVM)—with advanced transformer-based models, including BERT, RoBERTa, and DistilBERT. The methodology encompasses data collection, preprocessing, model training, and evaluation. Results show that transformer-based models, while computationally intensive, offer competitive precision scores compared to traditional methods. Specifically, BERT-base-uncased achieved a precision of 0.949 with a total training time of 4720.6 seconds. In comparison, traditional methods like Multinomial Naive Bayes, Logistic Regression, and SVM exhibited precisions of 0.917, 0.966, and 0.965, respectively, with significantly lower training times of 5.48, 19.86, and 61.73 seconds. RoBERTa and DistilBERT, other transformer-based models, achieved precisions of 0.961 and 0.962, respectively, with training times of 4488.35 and 952.34 seconds. These results highlight the trade-off between computational cost and performance in advanced NLP models. Traditional models demonstrated faster training times but slightly lower precision, emphasizing the need for balancing efficiency and accuracy in real-world applications. This research underscores the potential of transformer-based models to enhance the organization and retrieval of information in large scientific datasets, offering valuable insights.