Extracting Structure Information from Narrative Medical Reports based on LLMs

Extracting structured information and key details from medical report narratives is crucial to support healthcare data management, analysis and decision-making. However, the specialized nature of the reports, the complexity of the contents, and the high accuracy requirements of the results pose significant challenges to the structuring task. In this paper, we develop an LLM-based method to extract structure information from medical report narratives. Defining the structuring problem as mapping the narrative reports to the domain ontology, we design a framework to develop specialized LLMs that automatically learn and establish the mappings. At the core of this framework are report partitioning and interactive training data generation modules are. By separating complete reports into logically independent segments and training the LLMs on these segments independently, the trained LLMs can accurately capture the semantic relationships within each segment. Additionally, we explore different LLMs and formulate a simplistic scoring method to compare their accuracy, enabling us to select the best-performing model. Experimental evaluation on a real-world breast ultrasound report dataset demonstrates that our method achieves high accuracy with a small training dataset (400 samples). Specifically, the accuracy of structural information extraction and the attribute-value matching accuracy both exceed 96%.