AI-Simulated User Agents for Design Research: A Human–AI Collaborative Framework Leveraging Large Language Models

Traditional user research in design involves persistent tensions among scale, depth, and resource demands. We propose a three-tier conceptual framework for using large language models as simulated-user aids in design research. The evidence is separated by status: Studies 1 and 2 are company-confirmed aggregate commercial summaries and are reported descriptively, whereas Study 3 is an exploratory same-model prompt-format pilot. Study 3 compared a generic prompt with a framework-inspired structured prompt on 24 briefs using 14 included human-evaluator records from a mixed convenience sample and 140 evaluator–report ratings. The primary crossed mixed models found no condition coefficient at p < 0.05 and no Holm-adjusted advantage. Task-level Wilcoxon sensitivity results were likewise non-significant after Holm’s adjustment; forced choice was 5 structured versus 7 generic among 12 decisive responses (exact p = 0.774). The evidence therefore supports a bounded conceptual workflow and preliminary subjective report appraisal.

Authors

Institutions

Publication Details

Journal
International Journal of Human-Computer Interaction
Published
2026-10-13
DOI
https://doi.org/10.1080/10447318.2026.2741021
Primary Topic
Usability and User Interface Design
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

AI-Simulated User Agents for Design Research: A Human–AI Collaborative Framework Leveraging Large Language Models

Yuying Pei, Ling Fan
International Journal of Human-Computer Interaction
Usability and User Interface Design
article

AI-Simulated User Agents for Design Research: A Human–AI Collaborative Framework Leveraging Large Language Models

Yuying Pei, Ling Fan
article en

Abstract

Traditional user research in design involves persistent tensions among scale, depth, and resource demands. We propose a three-tier conceptual framework for using large language models as simulated-user aids in design research. The evidence is separated by status: Studies 1 and 2 are company-confirmed aggregate commercial summaries and are reported descriptively, whereas Study 3 is an exploratory same-model prompt-format pilot. Study 3 compared a generic prompt with a framework-inspired structured prompt on 24 briefs using 14 included human-evaluator records from a mixed convenience sample and 140 evaluator–report ratings. The primary crossed mixed models found no condition coefficient at p < 0.05 and no Holm-adjusted advantage. Task-level Wilcoxon sensitivity results were likewise non-significant after Holm’s adjustment; forced choice was 5 structured versus 7 generic among 12 decisive responses (exact p = 0.774). The evidence therefore supports a bounded conceptual workflow and preliminary subjective report appraisal.

International Journal of Human-Computer Interaction
Tongji University (CN)
Openalex Percentile: Top 8%
Usability and User Interface Design
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.