A RETRIEVAL-AUGMENTED GENERATION FRAMEWORK FOR EXPLAINABLE ACADEMIC PAPER QUALITY ASSESSMENT

With the exponential growth of global scholarly output, traditional academic paper evaluation methods face significant challenges in reliability, consistency, and scalability. Peer review processes suffer from low inter-rater agreement and lengthy decision times, while bibliometric approaches systematically disadvantage emerging fields. To address these systemic limitations, this study proposes a novel evaluation framework leveraging Retrieval-Augmented Generation (RAG) architecture and large language models (LLMs). The framework implements a four-dimensional assessment mechanism—analyzing research questions, methodologies, results, and conclusions—supported by contextual knowledge retrieval and explainable judgment generation. Experimental validation demonstrates the superiority of the RAG-based approach over both human experts and conventional machine learning baselines, achieving an F1-score of 0.77 at the quartile level. Additionally, the system provides transparent evaluative judgments supported by comparable evidence from prior literature. This work contributes to advancing scholarly communication by offering a scalable, explainable, and reliable alternative to existing evaluation paradigms.