A Better Predictor of People Is Not a Better Synthetic Person: What Compute Scaling Laws for LLM Social Simulation Measure, Why Hyper-Rational and Flattened Simulators Mostly Measure Something Else, and the Deviation Test That Would Separate Knowing

(c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0. Life of Research Language models are proposed as stand-ins for survey respondents and experimental participants, and a 2026 study asked whether the current scaling paradigm will make them faithful enough. Across 85 base models trained under fixed compute budgets, it found that loss on opinion, behaviour and longitudinal simulation tasks falls log-linearly with compute, and it concluded that scale will improve social simulation in most settings. A separate literature reports the opposite tendency in more capable models: answers to crowd-estimation questions that are unhumanly exact, intuitive errors that vanish, risky choices predicted as more rational than people make them, and opinion distributions collapsed onto one option. In this analysis the author reviews and synthesises published results to show that the two literatures mostly measure different quantities under the one word "fidelity". The scaling evidence scores a model as a predictor: the probability it assigns to recorded human responses, mostly in base models. Most of the contrary evidence scores a model as a participant whose sampled behaviour is compared with people's, mostly after post-training, and much of it concerns the part of human behaviour that departs from the normative or knowledge-best answer. The author derives that top-1 accuracy and typicality tests cannot reward dispersion, that a perfectly modal responder beats the uniform baseline on a binary item exactly when the human majority exceeds 75 percent, and that wherever the human mode coincides with the normative option, knowing the answer is indistinguishable from knowing people. Two results do reach the predictor role in base models: reading-time fit that worsens as perplexity falls, which the split accommodates only through an untested conjecture, and survey responses that become near-uniform under randomised option order irrespective of size, which it does not explain. The paper consolidates 31 published results in a table and a figure, re-reads the source study's own exceptions, gives a reading procedure, and specifies a deviation-weighted evaluation that would separate the two kinds of gain.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-07
DOI
https://doi.org/10.5281/zenodo.24024331
Primary Topic
Ethics and Social Impacts of AI
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

A Better Predictor of People Is Not a Better Synthetic Person: What Compute Scaling Laws for LLM Social Simulation Measure, Why Hyper-Rational and Flattened Simulators Mostly Measure Something Else, and the Deviation Test That Would Separate Knowing

Pranay M. Mahendrakar
1 links
Zenodo (CERN European Organization for Nuclear Research)
Ethics and Social Impacts of AI
preprint

A Better Predictor of People Is Not a Better Synthetic Person: What Compute Scaling Laws for LLM Social Simulation Measure, Why Hyper-Rational and Flattened Simulators Mostly Measure Something Else, and the Deviation Test That Would Separate Knowing

Pranay M. Mahendrakar
preprint en

Abstract

(c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0. Life of Research Language models are proposed as stand-ins for survey respondents and experimental participants, and a 2026 study asked whether the current scaling paradigm will make them faithful enough. Across 85 base models trained under fixed compute budgets, it found that loss on opinion, behaviour and longitudinal simulation tasks falls log-linearly with compute, and it concluded that scale will improve social simulation in most settings. A separate literature reports the opposite tendency in more capable models: answers to crowd-estimation questions that are unhumanly exact, intuitive errors that vanish, risky choices predicted as more rational than people make them, and opinion distributions collapsed onto one option. In this analysis the author reviews and synthesises published results to show that the two literatures mostly measure different quantities under the one word "fidelity". The scaling evidence scores a model as a predictor: the probability it assigns to recorded human responses, mostly in base models. Most of the contrary evidence scores a model as a participant whose sampled behaviour is compared with people's, mostly after post-training, and much of it concerns the part of human behaviour that departs from the normative or knowledge-best answer. The author derives that top-1 accuracy and typicality tests cannot reward dispersion, that a perfectly modal responder beats the uniform baseline on a binary item exactly when the human majority exceeds 75 percent, and that wherever the human mode coincides with the normative option, knowing the answer is indistinguishable from knowing people. Two results do reach the predictor role in base models: reading-time fit that worsens as perplexity falls, which the split accommodates only through an untested conjecture, and survey responses that become near-uniform under randomised option order irrespective of size, which it does not explain. The paper consolidates 31 published results in a table and a figure, re-reads the source study's own exceptions, gives a reading procedure, and specifies a deviation-weighted evaluation that would separate the two kinds of gain.

Zenodo (CERN European Organization for Nuclear Research)
Ethics and Social Impacts of AI
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon