From 245 and 520 to 6xx: An application of subject indexing of finance documents using named entity recognition

AbstractIndexing acts as a key to subject access of information resources. It tags documents with terms that represent the intellectual content of the documents. Various techniques have been used to perform subject indexing of resources for better information retrieval. The computerisation age has brought new prospects that reduce human labour, and automation has been one of them. One of the techniques, Natural Language Processing (NLP) has been useful for handling textual resources. This research has been performed to use an NLP technique, Named Entity Recognition (NER) to generate subject index entries for books of the Finance domain from the respective titles and summaries obtained from their bibliographic records. It compares the performance of three NER services, namely DBpedia Spotlight, Dandelion Entity Extraction and Stanford NLP by integrating the respective services with the open-source data wrangling software OpenRefine. For the purpose of comparison, the generated index terms for the books are compared with keywords suggested by a general user sample, the librarian-assigned subject headings, and the Library of Congress Subject Headings (LCSH) that acts as a golden standard list. A scoring framework has also been developed based on the semantic similarity of the terms with the lists. The results obtained showcased the better performance from Dandelion Entity Extraction. This research presents the pros and cons of the method.