Analyzing User Prompt Quality: Insights From Data

This research investigates the quality of user prompts in Large Language Models (LLMs), such as Chat-GPT, focusing on the impact of user experience and prompt characteristics on model performance. For this research, we prepared an online survey consisting of questions to collect data and, involving a total of 91 observations, took part in this research over 15 days. The study reveals that month of user experience and prompt satisfaction significantly enhance the ROUGE F1 scores, indicating better alignment between prompts and generated outputs. However, factors such as user background and output satisfaction show no significant impact. Responses were pre-processed and analyzed using statistical packages in Python. The findings underscore the importance of designing longer, well-structured prompts to optimize LLM performance, contributing to improved interactions and outputs in artificial intelligence applications.