Exploring large language models in enterprise modeling

Modeling is a crucial aspect of the development process in various engineering disciplines. The application of modeling in these fields typically encompasses various phases, including target setting, requirement elicitation, architecture specification, system design, and test case development. This paper focuses on the initial stages of systems development, with a specific focus on requirements engineering (RE) in enterprise modeling (EM). In particular, we examine the potential for domain experts to be replaced by artificial intelligence (AI) usage. The objective of this research is to contribute to a more comprehensive understanding of the limitations of large language models (LLMs). To this end, we employ a process from hospitality management and contrast the output of ChatGPT with that of a domain expert in an experiment. A second experiment was subsequently conducted in light of the assumption that the quality of responses can be enhanced by defining the expected output modeling language notation in the prompt and a metamodel for prompt engineering. The findings of this paper indicate that LLMs cannot replace domain experts in modeling the current situation of an enterprise. However, they can be employed as a supporting tool in EM. Moreover, the development of reusable prompts has emerged as both a viable and promising avenue for future research.