Semantic Embedding-Based Recommender System for Research Paper Discovery and Subject Area Prediction

This paper presents an innovative approach to research paper recommendation and subject area prediction, leveraging advanced natural language processing techniques. The research encompasses two primary sections: large-scale classification for subject area prediction using a shallow Multi-Layer Perceptron (MLP) model and research paper recommendation powered by a Sentence Transformer model. Through meticulous data cleaning and preprocessing, including tokenization and text vectorization, the research ensures high-quality embeddings that capture the semantic meaning and context of each research paper. The MLP model facilitates accurate prediction of subject areas based on paper titles and abstracts, while the Sentence Transformer model generates rich semantic embeddings for research papers. The system architecture involves training these models on a dataset of research paper abstracts and titles, achieving impressive performance metrics such as a categorical accuracy of 99.46% on the test set. The implementation of a user-friendly interface using Streamlit enhances accessibility, allowing users to input queries and receive personalized recommendations along with detailed paper categories. This transformative research not only advances the field of research paper recommendation but also contributes to the development of intelligent systems for knowledge discovery and academic research.