Calibrated reliance on generative-AI health advice: a perspective on correctness, risk, and escalation

Members of the public increasingly consult generative artificial intelligence (AI) for health advice, and research has largely asked whether they trust it. Trust is the wrong yardstick. Reliance on health advice is appropriate only when it tracks whether the advice is correct and what an error would cost. We argue for an outcome standard, calibrated reliance, under which four actions (adopt, verify, reject, and escalate to professional care) are judged jointly against advice correctness and the cost of error. Drawing on trust calibration, automation bias, algorithm aversion, advice taking, and heuristic models of credibility, we read a purposive set of human-participant studies against this standard. The studies show both reliance errors. Identical advice is discounted when labeled as AI-generated, which is consistent with underreliance. Low-accuracy AI responses are rated and followed much like physicians' responses, and lay users assisted by large language models choose care dispositions no better than controls, which is consistent with overreliance. Both patterns suggest that lay reliance responds to source and fluency cues more than to correctness. Interventions that require users to act, such as verification affordances, cognitive forcing, and expressed uncertainty, show more promise than passive labels, but no study has yet manipulated correctness and graded clinical risk together while observing verification and escalation. We state five propositions with their evidential status, operational definitions, and a factorial design that would test them.

Authors

Institutions

Publication Details

Journal
Frontiers in Public Health
Published
2026-11-18
DOI
https://doi.org/10.3389/fpubh.2026.1974040
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Calibrated reliance on generative-AI health advice: a perspective on correctness, risk, and escalation

Qiming Liu, Fangrong Cao, Ruolei Deng
1 links
Frontiers in Public Health
Artificial Intelligence in Healthcare and Education
article

Calibrated reliance on generative-AI health advice: a perspective on correctness, risk, and escalation

Qiming Liu, Fangrong Cao, Ruolei Deng
article en

Abstract

Members of the public increasingly consult generative artificial intelligence (AI) for health advice, and research has largely asked whether they trust it. Trust is the wrong yardstick. Reliance on health advice is appropriate only when it tracks whether the advice is correct and what an error would cost. We argue for an outcome standard, calibrated reliance, under which four actions (adopt, verify, reject, and escalate to professional care) are judged jointly against advice correctness and the cost of error. Drawing on trust calibration, automation bias, algorithm aversion, advice taking, and heuristic models of credibility, we read a purposive set of human-participant studies against this standard. The studies show both reliance errors. Identical advice is discounted when labeled as AI-generated, which is consistent with underreliance. Low-accuracy AI responses are rated and followed much like physicians' responses, and lay users assisted by large language models choose care dispositions no better than controls, which is consistent with overreliance. Both patterns suggest that lay reliance responds to source and fluency cues more than to correctness. Interventions that require users to act, such as verification affordances, cognitive forcing, and expressed uncertainty, show more promise than passive labels, but no study has yet manipulated correctness and graded clinical risk together while observing verification and escalation. We state five propositions with their evidential status, operational definitions, and a factorial design that would test them.

Frontiers in Public HealthVol. 14
Huzhou Normal University (CN)
Good health and well-being
Artificial Intelligence in Healthcare and Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon