Applying the design of experiments to Large Language Models

ChatGPT and Large Language Models (LLMs), in general, have become a growing source of information for students, practitioners, researchers, and the public. This article is on application of statistically designed experiments to the engineering of prompts so that we can learn from LLM platforms, beyond common direct queries. We introduce and demonstrate a methodology for setting up a designed sequence of prompts and apply text analytic methods, such as latent semantic analysis to analyze responses. The novelty is in the combination of the statistical design of experiments with text analytics to enhance the learning from LLM applications. This double helix model of learning focuses on evaluating factor effects with default options of LLM platforms. It can be implemented in any application domain and, from that perspective, can be combined in any data-driven study.