Evaluating and Improving Prompt Quality in LLM-Based Assistants: A Synthesis of Criteria and Indicators

Generative AI (GenAI) assistants, particularly large language models (LLMs), are gaining increasing relevance across domains. The quality of outputs generated by these systems is highly contingent on the input prompts, giving rise to new professional roles such as prompt engineers. In this study, we systematically examine evaluation criteria and optimization methods that can improve prompt quality. Drawing on a systematic literature review, we identify key criteria, including clarity, accuracy, and precision, and initial measurement techniques. In addition, we synthesize common optimization methods such as iterative refinement and shot-based prompting. Our work contributes to the growing efforts to standardize the evaluation and improvement of prompts in interactions with LLM-based assistants, thereby fostering a more rigorous and coherent understanding of the prompt quality construct.

Authors

Publication Details

Journal
ScholarSpace (University of Hawaii at Manoa)
Published
2025-12-23
Primary Topic
Topic Modeling
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Evaluating and Improving Prompt Quality in LLM-Based Assistants: A Synthesis of Criteria and Indicators

Mahei Manhai Li, Philipp Reinhard, Vladimir Sajzev, Jan Marco Leimeister
ScholarSpace (University of Hawaii at Manoa)
Topic Modeling
article

Evaluating and Improving Prompt Quality in LLM-Based Assistants: A Synthesis of Criteria and Indicators

Mahei Manhai Li, Philipp Reinhard, Vladimir Sajzev, Jan Marco Leimeister
article en

Abstract

Generative AI (GenAI) assistants, particularly large language models (LLMs), are gaining increasing relevance across domains. The quality of outputs generated by these systems is highly contingent on the input prompts, giving rise to new professional roles such as prompt engineers. In this study, we systematically examine evaluation criteria and optimization methods that can improve prompt quality. Drawing on a systematic literature review, we identify key criteria, including clarity, accuracy, and precision, and initial measurement techniques. In addition, we synthesize common optimization methods such as iterative refinement and shot-based prompting. Our work contributes to the growing efforts to standardize the evaluation and improvement of prompts in interactions with LLM-based assistants, thereby fostering a more rigorous and coherent understanding of the prompt quality construct.

ScholarSpace (University of Hawaii at Manoa)
Openalex Percentile: Top 18%
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.