When Should a Biomedical Prediction Lose Its Authority to Guide Care? Causal Identifiability, Safe Decision Frontiers, and Exploratory Falsification Tests

Background. Prediction accuracy after deployment need not measure the clinical effect of showing a prediction or recommendation, because display can change decisions, treatment, outcomes, and subsequent observations. We examine when a deployed recommendation can justifiably retain, lose, or regain authority under incomplete causal information.Methods. We distinguish the intention-to-display effect from treatment and prediction-performance estimands; restate three elementary analytical limits concerning missing counterfactuals, risk-constrained exposure, and unrestricted temporal change; and reanalyze two separately designed synthetic experiment series. A static study (v1.8A) evaluated nine scenario families, 400 replications per family, and sensitivity to transportability and a reference-policy loss constraint. A distinct six-period dynamic study (v1.9E) examined nine algorithms in 18 synthetic families (129,600 policy trajectories). An independent implementation of the static decision rule was checked against 129,600 original rule evaluations (v1.9F software audit).Results. In the static heterogeneous, informative-missingness family, a risk-constrained rule yielded regret of 19.10 per 1,000 eligible events versus 29.61 for a specified complete-case Bayesian rule; however, this result did not establish novelty over equivalently constrained safe policy learning and reversed in other scenarios. In the dynamic study, a simplified authority gate had higher mean loss than nominal-model Bayesian planning in 13/18 families and higher loss than an exposure-zero static proxy in 7/18 families. Its loss was 0.0963 versus 0.0001 under persistent harm for the static proxy, and 0.3784 versus 0.2431 under inverted signals. The v1.9F audit reproduced all static decisions but showed that the v1.9E static proxy was not the same policy.Interpretation. The experiments support a negative and conditional methodological conclusion: a defensible authority decision requires an identified estimand, a temporally credible evidence set, a legitimate action set, and comparisons with information- and permission-matched policies. The synthetic experiments neither establish clinical efficacy nor demonstrate an incremental advantage of the full KAFKA III architecture.

Authors

Publication Details

Journal
Preprints.org
Published
2026-11-18
DOI
https://doi.org/10.20944/preprints202611.1898.v1
Primary Topic
Advanced Causal Inference Techniques
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

When Should a Biomedical Prediction Lose Its Authority to Guide Care? Causal Identifiability, Safe Decision Frontiers, and Exploratory Falsification Tests

AQUILES ESAÚ DA SILVA REIS, Raphael Di Giorgio
Preprints.org
Advanced Causal Inference Techniques
preprint

When Should a Biomedical Prediction Lose Its Authority to Guide Care? Causal Identifiability, Safe Decision Frontiers, and Exploratory Falsification Tests

AQUILES ESAÚ DA SILVA REIS, Raphael Di Giorgio
preprint en

Abstract

Background. Prediction accuracy after deployment need not measure the clinical effect of showing a prediction or recommendation, because display can change decisions, treatment, outcomes, and subsequent observations. We examine when a deployed recommendation can justifiably retain, lose, or regain authority under incomplete causal information.Methods. We distinguish the intention-to-display effect from treatment and prediction-performance estimands; restate three elementary analytical limits concerning missing counterfactuals, risk-constrained exposure, and unrestricted temporal change; and reanalyze two separately designed synthetic experiment series. A static study (v1.8A) evaluated nine scenario families, 400 replications per family, and sensitivity to transportability and a reference-policy loss constraint. A distinct six-period dynamic study (v1.9E) examined nine algorithms in 18 synthetic families (129,600 policy trajectories). An independent implementation of the static decision rule was checked against 129,600 original rule evaluations (v1.9F software audit).Results. In the static heterogeneous, informative-missingness family, a risk-constrained rule yielded regret of 19.10 per 1,000 eligible events versus 29.61 for a specified complete-case Bayesian rule; however, this result did not establish novelty over equivalently constrained safe policy learning and reversed in other scenarios. In the dynamic study, a simplified authority gate had higher mean loss than nominal-model Bayesian planning in 13/18 families and higher loss than an exposure-zero static proxy in 7/18 families. Its loss was 0.0963 versus 0.0001 under persistent harm for the static proxy, and 0.3784 versus 0.2431 under inverted signals. The v1.9F audit reproduced all static decisions but showed that the v1.9E static proxy was not the same policy.Interpretation. The experiments support a negative and conditional methodological conclusion: a defensible authority decision requires an identified estimand, a temporally credible evidence set, a legitimate action set, and comparisons with information- and permission-matched policies. The synthetic experiments neither establish clinical efficacy nor demonstrate an incremental advantage of the full KAFKA III architecture.

Preprints.org
Advanced Causal Inference Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon