CLERK: A Companion Large Language Model Expert for modeling Regulatory Knowledge

Large Language Models (LLMs) have the potential to support the transformation of natural language legal text into a regulatory model, a task conventionally known to be time consuming and error prone when done manually. In this paper, we introduce CLERK: a C ompanion L LM E xpert for modeling R egulatory K nowledge existing in natural language legal texts. CLERK captures regulatory knowledge in the format of Legal Goal Requirements Language (GRL) models. CLERK offers three key contributions, utilizing established prompting techniques: (1) Adopting the Tree-of-Thought (ToT) prompting framework, CLERK streamlines the regulatory modeling process by breaking down complex steps into manageable tasks and focusing on those essential for constructing a Legal GRL model only. (2) The ToT framework enables self-evaluation of intermediate outputs. (3) CLERK enhances consistency and clarity, by leveraging additional in-context learning prompting techniques, such as few-shot prompting and output formatting with an explicit syntax definition. Experiments with eight regulatory articles from two domains (healthcare and energy communities) display a notable improvement brought about by CLERK compared to previous approaches. This improvement pertains to identifying relevant actors, goals and their deontic modalities, as well as the relationships among goals.