What Should We Engineer in Prompts? Training Humans in Requirement-Driven LLM Use
Prompting LLMs for complex tasks (e.g., building a trip advisor chatbot) needs humans to clearly articulate customized requirements (e.g., “start the response with a tl;dr”). However, existing prompt engineering instructions often lack focused training on requirement articulation and instead tend to emphasize increasingly automatable strategies (e.g., tricks like adding role-plays and “think step-by-step”). To address the gap, we introduce Requirement-Oriented Prompt Engineering ( ROPE ), a paradigm that focuses human attention on generating clear, complete requirements during prompting. We implement ROPE through an assessment and training suite that provides deliberate practice with LLM-generated feedback. In a randomized controlled experiment with 30 novices, ROPE significantly outperforms conventional prompt engineering training (20% vs. 1% gains), a gap that automatic prompt optimization cannot close. Furthermore, we demonstrate a direct correlation between the quality of input requirements and LLM outputs. Our work paves the way to empower more end-users to build complex LLM applications.
Authors
- Tongshuang Wu (ORCID: https://orcid.org/0000-0003-1630-0588)
- Qianou Ma (ORCID: https://orcid.org/0009-0002-8634-130X)
- Weirui Peng
- Kenneth R. Koedinger (ORCID: https://orcid.org/0000-0002-5850-4768)
- Hua Shen (ORCID: https://orcid.org/0000-0002-4928-525X)
- Chenyang Yang (ORCID: https://orcid.org/0000-0001-5016-7296)
Institutions
- University of Washington (US)
- Carnegie Mellon University (US)
- Columbia University (US)
Publication Details
- Journal
- ACM Transactions on Computer-Human Interaction
- Published
- 2025-04-24
- DOI
- https://doi.org/10.1145/3731756
- Citations
- 13
- Primary Topic
- Spreadsheets and End-User Computing
- Type
- article
- Field-Weighted Citation Impact
- 30.00