A Comparative Study of Large Language Models for Goal Model Extraction

User stories, expressed in snippets of natural language text, are commonly used to elicit stakeholder's needs in agile software development. Requirement engineers model user stories to interpret the relations among goals and requirements. Manual transformation of goal models has challenges such as, difficulty of converting lower-abstraction user stories into higher-level goals, and extraction of goals embedded in user stories depends on the skill of requirements engineers. In this paper we introduce a technique that leverages Large Language Models (LLMs) to automatically generate goal models from user stories. The approach uses Iterative Prompt Engineering that guides LLM to extract intentional elements and generate its XML-compatible representation in Goal-oriented Requirements Language (GRL). The generated models can be visualized using jUCMNav tool. We evaluated our approach using three LLMs: GPT-4, Llama and Cohere. Our qualitative evaluation indicates that GPT-4 or Llama can be used to assist requirements engineers in modeling as they can produce GRL goal models that are understandable. Additionally, these LLMs are capable of exposing soft goals that are not apparent to stakeholders who are new to the domain.