Can an LLM Learn an Unseen Engineering Language? An Empirical Study of Prompt Engineering on ChatGPT and Function Modeling

Abstract In the early stages of engineering design, designers build various models using different languages. Meanwhile, large language models (LLMs) have evolved to a point where they can produce human-like intelligent behavior in many areas where the conversations are in text. In addition, these LLMs can also learn new linguistic patterns at runtime, through prompt engineering. This development creates the possibility of teaching LLMs design modeling languages too, as long as the modeling language is text-based. To this end, this paper examines the extent to which an LLM (specifically Open AI’s GPT-3.5) can learn a previously unseen engineering modeling language (specifically the function modeling language). Four experiments are conducted to test the LLM’s learning abilities with progressively more complex learning challenges. The results show that the LLM can learn function modeling vocabularies and simple grammar rules to formulate functional sentences and apply them to both individual devices and systems. However, the LLM performs inconsistently when it is asked to include the topology of a function model in the text-based functional sentences.