Leveraging LLMs for Unstructured Direct Elicitation of Decision Rules

Abstract Unstructured Direct Elicitation (UDE) offers a flexible method to capture consumer preferences and decision rules in an unstructured format such as writing an email. However, it relies on subjective human coding and indicative consideration set sizes to make accurate predictions on consideration decisions. This research leverages large language models (LLMs) to replace human judges and make predictions without the need for additional information like indicative consideration set sizes. Empirical analyses show that fine-tuned LLMs effectively interpret decision rules and handle sophisticated considerations in a complex product scenario (automotive study), outperforming the best UDE models by capturing over 25% more information, while their performance in a moderate-scale study on mobile phones is comparable to the best UDE models. The use of LLMs enhances scalability, cost efficiency, and consistency in comprehending unstructured text data and making predictions, offering a promising alternative to human judges and enabling large-scale, real-time implementation of UDE in marketing research and practice. Together with their ability to interact with users, LLMs fine-tuned with representative datasets may serve as a valuable knowledgebase to summarize consumer preferences and decision rules and supply insights for the creation and simulation of marketing strategies.