A Continuous Quality Improvement Evaluation Approach for Extracting Legal Knowledge

Legal texts, particularly case precedents, are a rich source of legal knowledge that can be harnessed for intelligent legal work.The extraction of legal knowledge in the format of <head entity, relation, tail entity> is becoming an increasingly critical task within the legal domain.While deep learning-based named entity recognition and other approaches are available for extracting legal knowledge triplets from texts, effectively evaluating the quality of these extracted triplets remains a significant challenge.To tackle this challenge, this paper introduces a continuous quality improvement evaluation approach for the extraction of legal knowledge.This method involves segmenting the initial legal texts into several parts, employing natural language processing techniques to extract knowledge triplets from one segment at a time, and then manually evaluating these triplets.Correctly identified triplets are utilized to aid in the extraction process for subsequent segments.Through iterative application, this approach allows for the efficient and effective extraction of knowledge triplets from legal texts.An experimental study, utilizing 382 cases from the Caselaw Access Project, was undertaken to generate and evaluate legal knowledge triplets.The findings underscore the efficiency and effectiveness of the proposed approach in extracting legal knowledge triplets and significantly enhancing their accuracy.The extracted triplets could lay a foundational groundwork for constructing a legal knowledge graph.