Cognitive Mirrors: Do Large Language Models Inherit Human Cognitive Biases, and Can Metacognitive Prompting Reduce Them?
Large language models (LLMs) are increasingly used where consistent reasoning matters, yet it is unclear how far they reproduce the systematic judgment errors documented in human psychology, and whether telling a model about a bias helps it avoid that bias. I test five biases (anchoring, framing, sycophancy, the availability heuristic and confidence miscalibration) with a paired design in which the informed prompt names the bias and asks the model to resist it. Four open-weight models were queried (2,160 calls). One reasoning model returned a usable answer for only 30% of calls because its output was cut off by the token limit, so inference covers Llama 3.1 8B, Llama 3.3 70B and GPT-OSS 20B, with scenarios as the unit of analysis. A high anchor raised estimates by a factor of 1.9 on average (95% CI 1.1 to 3.2), ranging from 1.3 to 2.7 across models, and a gain frame raised the YES rate by 53 percentage points (CI 33 to 74). The explicit warning left both effects largely intact (anchoring factor 1.6, framing difference 54 points; paired 𝑝 = .15 and 𝑝 = .96). No model abandoned a correct answer under user pushback in 172 usable trials. Models stated 98 to 99% confidence at 85 to 90% accuracy, and the overconfidence concentrated in the hardest questions (31 points). Telling a model about a bias changes little of what it does. Prompts, outputs and analysis code are available on request.
Authors
- Zaynab Shahid
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-10-14
- DOI
- https://doi.org/10.5281/zenodo.23372108
- Primary Topic
- Decision-Making and Behavioral Economics
- Type
- preprint