AI background and advice uptake after low practice performance: A reproduction and exploratory extension of Horowitz and Kahn

Horowitz and Kahn (2024a) report a nonlinear association between AI background and answer switching after AI advice in a national-security-framed experiment. This study reproduces the focal background coefficients in their Table 2 model 7 to published precision and extends the analysis by separating opportunities to adopt correct and incorrect advice. The exploratory extension uses 18,560 AI-advice trials from 4,191 respondents who answered fewer than three of five practice questions correctly. At an AI-background index of 0.6 versus the respondent median, the primary quadratic model estimates 8.75 percentage points less beneficial-advice adoption and 10.93 points less harmful adoption. The beneficial estimate becomes less precise when the original covariates are restored, while harmful-adoption intervals remain below zero across those specifications. The adjusted net-accuracy comparison is inconclusive. Respondents with background of at least 0.6 nevertheless improve their observed accuracy by 3.74 points from initial to final answers. High-background support is sparse, all extension analyses are post hoc, and the historical log's cluster count remains unresolved. These results preserve the selected original association while qualifying what it establishes about the quality of advice use. They identify neither a causal effect of AI education nor general psychological calibration. This record contains the preprint and technical supplement for a selected-result reproduction and exploratory extension of Horowitz and Kahn (2024). The accompanying software and aggregate results are deposited separately. The original respondent data and authors' R/Stata scripts remain at the version 1.0 Harvard Dataverse deposit. The extension was not preregistered, and the manuscript distinguishes internal verification from independent CODECHECK certification.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-16
DOI
https://doi.org/10.5281/zenodo.24251489
Primary Topic
Decision-Making and Behavioral Economics
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

AI background and advice uptake after low practice performance: A reproduction and exploratory extension of Horowitz and Kahn

Dennis Phua
Zenodo (CERN European Organization for Nuclear Research)
Decision-Making and Behavioral Economics
preprint

AI background and advice uptake after low practice performance: A reproduction and exploratory extension of Horowitz and Kahn

Dennis Phua
preprint en

Abstract

Horowitz and Kahn (2024a) report a nonlinear association between AI background and answer switching after AI advice in a national-security-framed experiment. This study reproduces the focal background coefficients in their Table 2 model 7 to published precision and extends the analysis by separating opportunities to adopt correct and incorrect advice. The exploratory extension uses 18,560 AI-advice trials from 4,191 respondents who answered fewer than three of five practice questions correctly. At an AI-background index of 0.6 versus the respondent median, the primary quadratic model estimates 8.75 percentage points less beneficial-advice adoption and 10.93 points less harmful adoption. The beneficial estimate becomes less precise when the original covariates are restored, while harmful-adoption intervals remain below zero across those specifications. The adjusted net-accuracy comparison is inconclusive. Respondents with background of at least 0.6 nevertheless improve their observed accuracy by 3.74 points from initial to final answers. High-background support is sparse, all extension analyses are post hoc, and the historical log's cluster count remains unresolved. These results preserve the selected original association while qualifying what it establishes about the quality of advice use. They identify neither a causal effect of AI education nor general psychological calibration. This record contains the preprint and technical supplement for a selected-result reproduction and exploratory extension of Horowitz and Kahn (2024). The accompanying software and aggregate results are deposited separately. The original respondent data and authors' R/Stata scripts remain at the version 1.0 Harvard Dataverse deposit. The extension was not preregistered, and the manuscript distinguishes internal verification from independent CODECHECK certification.

Zenodo (CERN European Organization for Nuclear Research)
Decision-Making and Behavioral Economics
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon