Rethinking Think-Aloud in the Age of Language Models

Understanding the human mind has long been a central goal of cognitive science, yet behavioral measures such as button presses provide only indirect traces of thought. Think-Aloud protocols, where participants verbalize their reasoning, offer a more direct probe but have long been criticized as unreliable and difficult to scale. Here we revisit this debate using one of the largest Think-Aloud datasets to date—44,038 paired transcripts and choices in risky decision-making. Our results make three contributions. First, concurrent Think-Aloud analyzed with large language models (LLMs) surpasses classic cognitive models and neural networks in predicting behavior, showing that verbal reports contain generative information beyond behavior alone. Second, Think-Aloud generalizes across individuals with only a few examples, reflecting shared cognitive patterns. Third, Think-Aloud guides LLMs toward more human-like reasoning. Together, these findings position Think-Aloud as a scalable complement to behavioral measures, reopening a foundational debate in cognitive science and informing human-aligned AI.

Authors

Institutions

Publication Details

Published
2025-08-31
DOI
https://doi.org/10.31234/osf.io/6ta3z_v1
Citations
1
Primary Topic
Natural Language Processing Techniques
Type
article
Field-Weighted Citation Impact
2.48

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Rethinking Think-Aloud in the Age of Language Models

Hua-Dong Xiong, Hanbo Xie, Robert C. Wilson
1 citations
Natural Language Processing Techniques
2.48
article

Rethinking Think-Aloud in the Age of Language Models

Hua-Dong Xiong, Hanbo Xie, Robert C. Wilson
article en
1 citations

Abstract

Understanding the human mind has long been a central goal of cognitive science, yet behavioral measures such as button presses provide only indirect traces of thought. Think-Aloud protocols, where participants verbalize their reasoning, offer a more direct probe but have long been criticized as unreliable and difficult to scale. Here we revisit this debate using one of the largest Think-Aloud datasets to date—44,038 paired transcripts and choices in risky decision-making. Our results make three contributions. First, concurrent Think-Aloud analyzed with large language models (LLMs) surpasses classic cognitive models and neural networks in predicting behavior, showing that verbal reports contain generative information beyond behavior alone. Second, Think-Aloud generalizes across individuals with only a few examples, reflecting shared cognitive patterns. Third, Think-Aloud guides LLMs toward more human-like reasoning. Together, these findings position Think-Aloud as a scalable complement to behavioral measures, reopening a foundational debate in cognitive science and informing human-aligned AI.

University of Arizona (US)
Georgia Institute of Technology, Research Corporation for Scientific Advancement
Openalex Percentile: Top 9%
Natural Language Processing Techniques
2.48
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Rethinking Think-Aloud in the Age of Language Models — Hua-Dong Xiong, Hanbo Xie, et al. · (2025) | TGRS Research Map | TGRS