Ranking method for an academic paper search with an emphasis on comprehensiveness

Abstract In an academic paper search to confirm the novelty of a research project, it is important to improve the recall score for the number of search results that users can check to comprehensively collect research papers related to the user’s information need. However, it may be necessary to check papers in the lower ranks to cover the relevant papers comprehensively when using single search method. To improve the recall score for the number of papers that users can check, we considered that it would be effective to integrate ranking results using multiple search methods with different approaches. This is based on the idea that relevant papers that do not appear in the higher ranks for one method can be found using other methods, and that effect would be amplified by integrating more ranking results. As the methods to be integrated, we used the ranking methods in the vector space model, the query likelihood model, and a newly proposed method. Our method is based on a topic-based Boolean search that uses the topic analysis result from latent Dirichlet allocation, and ranked papers in descending order the number of times they are included in each search result. We performed an evaluation using the NTCIR- 1 and - 2 datasets, and confirmed that our topic-based search method showed different trends from rankings based on conventional vector space model and query likelihood model. Furthermore, we showed the best performance by the re-ranking using these three methods comparing other combinations.