Evaluation of Large Language Model Responses for 32 Diverse Personality Types Using the Best Worst Method (BWM)

Today, Chat Generative Pre-trained Transformer (GPT), a generative language model tool created by OpenAI in November 2022, has greatly impacted the AI field. Large language models (LLMs) effectively gather useful information from interaction with humans in diverse fields, including tutoring, translation, customer support, etc. Certainly, ChatGPT is a chatbot designed to be used in conversation to generate human-like responses, and it can answer all kinds of questions in different fields. This study aims to analyze the performance of LLMs by examining how they respond to different prompts based on individual preferences. We applied different personalities and car features to the prompts and employed the Best-Worst Method (BWM). In This approach, ChatGPT generated responses regarding which car features were considered best and worst based on the 32 different personality types. We evaluated the LLM's responses and examined its performance by categorizing all ChatGPT outputs. We categorize them based on each feature chosen and by which type of 32 personality is the best or least feature. This approach enables us to observe how various personalities of humans influence the responses generated by the language models. It will also show us how ChatGPT's decision-making process makes sense to experts in the field of psychology.