AI-Simulated User Agents for Design Research: A Human–AI Collaborative Framework Leveraging Large Language Models
Traditional user research in design involves persistent tensions among scale, depth, and resource demands. We propose a three-tier conceptual framework for using large language models as simulated-user aids in design research. The evidence is separated by status: Studies 1 and 2 are company-confirmed aggregate commercial summaries and are reported descriptively, whereas Study 3 is an exploratory same-model prompt-format pilot. Study 3 compared a generic prompt with a framework-inspired structured prompt on 24 briefs using 14 included human-evaluator records from a mixed convenience sample and 140 evaluator–report ratings. The primary crossed mixed models found no condition coefficient at p < 0.05 and no Holm-adjusted advantage. Task-level Wilcoxon sensitivity results were likewise non-significant after Holm’s adjustment; forced choice was 5 structured versus 7 generic among 12 decisive responses (exact p = 0.774). The evidence therefore supports a bounded conceptual workflow and preliminary subjective report appraisal.
Authors
- Yuying Pei (ORCID: https://orcid.org/0009-0009-7810-4831)
- Ling Fan
Institutions
- Tongji University (CN)
Publication Details
- Journal
- International Journal of Human-Computer Interaction
- Published
- 2026-10-13
- DOI
- https://doi.org/10.1080/10447318.2026.2741021
- Primary Topic
- Usability and User Interface Design
- Type
- article
- Field-Weighted Citation Impact
- 0.00