Using Large Language Models for Research Paper Categorization: Insights from Abstract-Based Analysis in Transportation Research

The volume of transportation research publications has grown rapidly in recent years, making it increasingly difficult for practitioners and scholars to efficiently identify and categorize relevant (ML) techniques. This paper explores the use of transformer-based large language models (LLMs) and a variety of classical models for efficient classification and categorization of research papers, specifically in the field of transportation, based on their abstracts, with selected experiments also incorporating paper titles. In this study, the term LLMs is used broadly to refer to large pretrained transformer-based language models, including encoder-based architectures such as BERT (Bidirectional Encoder Representations from Transformers), RoBERTa (Robustly Optimized BERT Pretraining Approach), and DeBERTa (Decoding-enhanced BERT with disentangled attention), which are utilized in this paper. Selected LLMs, including BERT-like models, are fine-tuned and evaluated on a benchmark dataset covering a range of transportation application categories. The experimental design includes multiple scenarios, including zero-shot, few-shot, fine-tuning, and feature extraction pipelines. The results demonstrate that the use of LLMs for document and paper classification achieves substantially higher accuracy than traditional ML approaches, and indicate the potential of LLMs to support automated document classification tasks for digital libraries.