Where do LLM-simulated patients with delusions fail to reproduce human findings? Preregistration v0.16 and frozen materials

Preregistration (v0.16) and frozen materials for a falsification test of LLM-simulated patients with delusions: on held-out human findings (sampling-task draws to decision, U2; baseline conviction and belief-revision interview items, U5), do simulated patients fail to reproduce them, and in which direction? Claims are limited to behavioural fidelity, not mechanism. Contents: - preregistration_v0.16.md: the preregistration (SHA-256 93fa1dca2cff9bd672a0fc18f5456e4d5bb4e89fea67bae5dc7e9026301899ed). - appendix_R.md: frozen rule texts R1 to R17, describing the hashed harness code (SHA-256 f96d089fe3a93676d489c4aefaf6e187f04235789b1bac3d5554f3495ffa5353). - appendix_T_frozen_texts.md, appendix_A_generation_prompts.md, persona_assembly_spec.md: frozen patient-facing and rater texts, persona generation prompts, persona assembly specification. - verdict.py (SHA-256 ba0dc21bb89cf36e6e616f85fa4a3aad75a8c7024d0c8bc5ab87923b5ca07c26): the confirmatory verdict script, which holds the bands and decides every verdict. - verdict_selftest.py (SHA-256 67a8a4ca148a813d128c1647d4d6edb4c8a10e8e08a1872b2e1c31b3ec2de738) and verdict_selftest_results.txt: self-test under the faithful null. - power_check.py (SHA-256 d8c07c1736cfa886dd9df52f0e477213bc3380c8c5952f81320ceb08f476d25f) and power_check_results.txt: descriptive power check. - v07_u2, v08_review5, v07b_shared_persons, v09_f, nullsim (.py and _results.txt): false-failure analyses cited in section 1. - harness_3109381.zip: the study harness at the posted commit 310938177a941c6f56087a6e6fbd7ef4d9e73939 (archive SHA-256 fa29d6ed5cc3fe29f6dfce430021bc381c9b32976809d0e75499c395674479ab). - decision_log.md: dated record of every design decision, including errors and corrections. - osf_posting_checklist.md: the dated public posts the preregistration commits to during the study. Files referred to in the preregistration as analysis/… are at the top level of this record. The held-out test set (test_targets.md, SHA-256 in the preregistration header) is withheld and released with the results. No study data exist at the time of this deposit: no patient-model study call has been made. Amendment 1 (2026-11-18): frozen persona file hash recorded before any patient-model call; see amendment_01_personas.md Amendment 2 (2026-11-18): Ollama server context set to 16,384 before the first study call; see amendment_02_olmo_run_setup.md.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-19
DOI
https://doi.org/10.5281/zenodo.24358002
Citations
1
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
article
Field-Weighted Citation Impact
0.97
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Where do LLM-simulated patients with delusions fail to reproduce human findings? Preregistration v0.16 and frozen materials

Arnav Amit
1 citations
Zenodo (CERN European Organization for Nuclear Research)
Artificial Intelligence in Healthcare and Education
0.97
article

Where do LLM-simulated patients with delusions fail to reproduce human findings? Preregistration v0.16 and frozen materials

Arnav Amit
article en
1 citations

Abstract

Preregistration (v0.16) and frozen materials for a falsification test of LLM-simulated patients with delusions: on held-out human findings (sampling-task draws to decision, U2; baseline conviction and belief-revision interview items, U5), do simulated patients fail to reproduce them, and in which direction? Claims are limited to behavioural fidelity, not mechanism. Contents: - preregistration_v0.16.md: the preregistration (SHA-256 93fa1dca2cff9bd672a0fc18f5456e4d5bb4e89fea67bae5dc7e9026301899ed). - appendix_R.md: frozen rule texts R1 to R17, describing the hashed harness code (SHA-256 f96d089fe3a93676d489c4aefaf6e187f04235789b1bac3d5554f3495ffa5353). - appendix_T_frozen_texts.md, appendix_A_generation_prompts.md, persona_assembly_spec.md: frozen patient-facing and rater texts, persona generation prompts, persona assembly specification. - verdict.py (SHA-256 ba0dc21bb89cf36e6e616f85fa4a3aad75a8c7024d0c8bc5ab87923b5ca07c26): the confirmatory verdict script, which holds the bands and decides every verdict. - verdict_selftest.py (SHA-256 67a8a4ca148a813d128c1647d4d6edb4c8a10e8e08a1872b2e1c31b3ec2de738) and verdict_selftest_results.txt: self-test under the faithful null. - power_check.py (SHA-256 d8c07c1736cfa886dd9df52f0e477213bc3380c8c5952f81320ceb08f476d25f) and power_check_results.txt: descriptive power check. - v07_u2, v08_review5, v07b_shared_persons, v09_f, nullsim (.py and _results.txt): false-failure analyses cited in section 1. - harness_3109381.zip: the study harness at the posted commit 310938177a941c6f56087a6e6fbd7ef4d9e73939 (archive SHA-256 fa29d6ed5cc3fe29f6dfce430021bc381c9b32976809d0e75499c395674479ab). - decision_log.md: dated record of every design decision, including errors and corrections. - osf_posting_checklist.md: the dated public posts the preregistration commits to during the study. Files referred to in the preregistration as analysis/… are at the top level of this record. The held-out test set (test_targets.md, SHA-256 in the preregistration header) is withheld and released with the results. No study data exist at the time of this deposit: no patient-model study call has been made. Amendment 1 (2026-11-18): frozen persona file hash recorded before any patient-model call; see amendment_01_personas.md Amendment 2 (2026-11-18): Ollama server context set to 16,384 before the first study call; see amendment_02_olmo_run_setup.md.

Zenodo (CERN European Organization for Nuclear Research)
Openalex Percentile: Top 26%
Artificial Intelligence in Healthcare and Education
0.97
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon