Does Reasoning Give a Language Model a Personality? Within-Model Effects of Thinking on Big Five Trait Scores and Construct Validity.

Many current language models generate a chain of "reasoning" tokens before answering, and many expose this as a toggle. Does turning reasoning on change how a model responds to Big Five personality questionnaires? This study uses a within-model design: the same model answers the same items with thinking off and with thinking on. Four public-domain IPIP Big Five inventories (the 20-item Mini-IPIP, the 50-item IPIP Factor Markers, the 100-item Big Five Aspect Scales, and the 120-item IPIP-NEO, 290 items in total) were administered to ten open-weight hybrid models with a reasoning toggle, one reasoning-native open model (OLMo 3, which cannot disable thinking), and one commercial model (Claude Haiku 4.5, administered through its provider's Batch API). The think-off baseline for the ten hybrids reuses direct-answer data from a prior study; that reuse is validated directly, as a fresh think-off run reproduces the prior domain means to within 0.07 points on the five-point scale, against reasoning effects 5 to 28 times larger. Two findings emerge. (1) Reasoning shifts reported trait scores substantially and in a consistent direction: models present as more emotionally stable (about +0.9 on a 5-point scale) and less extraverted (about -0.6 to -0.8). The two largest effects hold across open-weight and commercial models; the smaller conscientiousness and agreeableness effects do not generalize and are not claimed. Convergent validity rises under reasoning (mean convergent r 0.65 to 0.84) while heterotrait correlation is flat, so the convergent-discriminant gap widens because same-trait agreement rises, not because traits separate. (2) Reasoning does not create within-model coherence: treating repeated generations as respondents, internal-consistency reliability stays near zero with thinking off and on (median alpha near zero; zero of 200 hybrid and zero of 20 commercial model-by-domain cells reach 0.70), and the same holds for the reasoning-native model that cannot stop thinking. Reasoning changes what a model says about its personality without making that personality cohere as an individual-level property. This deposit contains the full manuscript (PDF, LaTeX source, and a readable Markdown version), a plain-language summary, the raw response databases, all administration and analysis code, the four instrument files, and every derived table. The analysis scripts regenerate every result deterministically. The software in the bundle is released under the MIT License; the deposit license (CC-BY-4.0) applies to the manuscript and data.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-06-27
DOI
https://doi.org/10.5281/zenodo.20974668
Primary Topic
Personality Traits and Psychology
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Does Reasoning Give a Language Model a Personality? Within-Model Effects of Thinking on Big Five Trait Scores and Construct Validity.

Trevor Johnson
Zenodo (CERN European Organization for Nuclear Research)
Personality Traits and Psychology
preprint

Does Reasoning Give a Language Model a Personality? Within-Model Effects of Thinking on Big Five Trait Scores and Construct Validity.

Trevor Johnson
preprint en

Abstract

Many current language models generate a chain of "reasoning" tokens before answering, and many expose this as a toggle. Does turning reasoning on change how a model responds to Big Five personality questionnaires? This study uses a within-model design: the same model answers the same items with thinking off and with thinking on. Four public-domain IPIP Big Five inventories (the 20-item Mini-IPIP, the 50-item IPIP Factor Markers, the 100-item Big Five Aspect Scales, and the 120-item IPIP-NEO, 290 items in total) were administered to ten open-weight hybrid models with a reasoning toggle, one reasoning-native open model (OLMo 3, which cannot disable thinking), and one commercial model (Claude Haiku 4.5, administered through its provider's Batch API). The think-off baseline for the ten hybrids reuses direct-answer data from a prior study; that reuse is validated directly, as a fresh think-off run reproduces the prior domain means to within 0.07 points on the five-point scale, against reasoning effects 5 to 28 times larger. Two findings emerge. (1) Reasoning shifts reported trait scores substantially and in a consistent direction: models present as more emotionally stable (about +0.9 on a 5-point scale) and less extraverted (about -0.6 to -0.8). The two largest effects hold across open-weight and commercial models; the smaller conscientiousness and agreeableness effects do not generalize and are not claimed. Convergent validity rises under reasoning (mean convergent r 0.65 to 0.84) while heterotrait correlation is flat, so the convergent-discriminant gap widens because same-trait agreement rises, not because traits separate. (2) Reasoning does not create within-model coherence: treating repeated generations as respondents, internal-consistency reliability stays near zero with thinking off and on (median alpha near zero; zero of 200 hybrid and zero of 20 commercial model-by-domain cells reach 0.70), and the same holds for the reasoning-native model that cannot stop thinking. Reasoning changes what a model says about its personality without making that personality cohere as an individual-level property. This deposit contains the full manuscript (PDF, LaTeX source, and a readable Markdown version), a plain-language summary, the raw response databases, all administration and analysis code, the four instrument files, and every derived table. The analysis scripts regenerate every result deterministically. The software in the bundle is released under the MIT License; the deposit license (CC-BY-4.0) applies to the manuscript and data.

Zenodo (CERN European Organization for Nuclear Research)
Reduced inequalities
Personality Traits and Psychology
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.