AN NLP ENRICHED SEARCH ENGINE FOR PATHOLOGY REPORTS

This dissertation explores the integration of Natural Language Processing (NLP) techniques toenrich and enhance the searchability and utility of pathology reports within the healthcare researchdomain. Pathology reports, which provide detailed insights into the diagnosis and progression ofdiseases, particularly cancer, are often composed of unstructured text. This presents significantchallenges in efficiently retrieving and analyzing relevant data. The study aims to develop and evaluatean NLP-enriched search engine tailored for pathology reports, leveraging various text enrichmenttechniques such as Named Entity Recognition (NER), concept mapping, and negation detection.The research problem addresses the need for robust search tools that can handle the complexityand variability of pathology reports. By implementing a comprehensive NLP pipeline, this dissertationdemonstrates the effectiveness of advanced text processing methods in improving information retrievalmetrics such as precision, recall, and F1 score. The methodology includes the processing of pathologyreports from the UNC Health system, the application of NLP models using scispaCy pipeline and the useof Apache Solr for indexing and search functionalities.Key contributions of this work include the development of enhancing cohort discovery fortranslational research and providing methodological insights into the relative effectiveness of various NLPtechniques. The results indicate significant improvements in search performance metrics, particularlywhen combining multiple enrichment strategies. These findings have broader implications for the field ofmedical informatics, offering a roadmap for future research in leveraging NLP to enhance the usability ofcomplex medical texts.This dissertation highlights the transformative potential of NLP in healthcare, particularly in thecontext of pathology reports. By improving the accessibility and accuracy of critical clinical information,the proposed NLP-enriched search engine has the potential to significantly enhance patient care andsupport ongoing medical research.