Source recovery does not guarantee inferential closure: Three lessons for auditing clinical AI claims

Methodological commentary; preprint, not peer reviewed. Background Recovering missing information from a clinical trial report may answer an audit question without supporting every inference drawn from the result. Source recovery, closure of the question as originally specified and evidentiary sufficiency for another use need to be distinguished. Main body This Comment uses three historical source-completion episodes from 18 purposefully assembled development archives. In a childhood-asthma trial, recovered group headers resolved the binding of counts to randomised groups, while eligible and observable participation denominators remained unclear. Programme entry did not directly measure access barriers, but this limitation did not refute a conditional explanation about possible access improvements. In a hypotension trial, table footnotes identified the estimator for a reported difference; this estimate should not be replaced by subtraction of the displayed medians. Repeated-alert aggregation and patient-level interpretation remained unresolved. In a brachytherapy-planning trial, the published layout restored readable symbols but exposed conflicting P-value directions. Reported elapsed planning times covered a fixed phase and did not identify active labour or whole-team savings. Disagreement about closure of the original reading question remains unresolved. The discussion addresses ordinary critical appraisal, moving the goalposts, excessive withholding and documentation burden. The cases are illustrative, not a representative survey, and do not validate an audit method or estimate its causal effect. Conclusions An audit can retain a supported observation, state which question a source repair answers and specify the limitation affecting another intended inference. Reporting these judgments separately avoids treating an apparently complete record as sufficient evidence for a clinical conclusion. AI assistance and human contribution OpenAI Codex agents were used for documentary synthesis of archived evidence, drafting, revision, source navigation, editorial checks, translation, language editing, journal-format adaptation and document preparation. The author directed the project, substantively reviewed the principal arguments, key supporting evidence and limitations, approved the final manuscript and takes responsibility for its content. The source-recovery episodes are illustrative historical reconstructions, not a representative survey, a new method experiment or a validation of clinical accuracy. Funding This work was self-funded; no external funding was received. Competing interests The author declares no competing interests.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-21
DOI
https://doi.org/10.5281/zenodo.24410914
Primary Topic
Meta-analysis and systematic reviews
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Source recovery does not guarantee inferential closure: Three lessons for auditing clinical AI claims

Chuan Yin
Zenodo (CERN European Organization for Nuclear Research)
Meta-analysis and systematic reviews
preprint

Source recovery does not guarantee inferential closure: Three lessons for auditing clinical AI claims

Chuan Yin
preprint en

Abstract

Methodological commentary; preprint, not peer reviewed. Background Recovering missing information from a clinical trial report may answer an audit question without supporting every inference drawn from the result. Source recovery, closure of the question as originally specified and evidentiary sufficiency for another use need to be distinguished. Main body This Comment uses three historical source-completion episodes from 18 purposefully assembled development archives. In a childhood-asthma trial, recovered group headers resolved the binding of counts to randomised groups, while eligible and observable participation denominators remained unclear. Programme entry did not directly measure access barriers, but this limitation did not refute a conditional explanation about possible access improvements. In a hypotension trial, table footnotes identified the estimator for a reported difference; this estimate should not be replaced by subtraction of the displayed medians. Repeated-alert aggregation and patient-level interpretation remained unresolved. In a brachytherapy-planning trial, the published layout restored readable symbols but exposed conflicting P-value directions. Reported elapsed planning times covered a fixed phase and did not identify active labour or whole-team savings. Disagreement about closure of the original reading question remains unresolved. The discussion addresses ordinary critical appraisal, moving the goalposts, excessive withholding and documentation burden. The cases are illustrative, not a representative survey, and do not validate an audit method or estimate its causal effect. Conclusions An audit can retain a supported observation, state which question a source repair answers and specify the limitation affecting another intended inference. Reporting these judgments separately avoids treating an apparently complete record as sufficient evidence for a clinical conclusion. AI assistance and human contribution OpenAI Codex agents were used for documentary synthesis of archived evidence, drafting, revision, source navigation, editorial checks, translation, language editing, journal-format adaptation and document preparation. The author directed the project, substantively reviewed the principal arguments, key supporting evidence and limitations, approved the final manuscript and takes responsibility for its content. The source-recovery episodes are illustrative historical reconstructions, not a representative survey, a new method experiment or a validation of clinical accuracy. Funding This work was self-funded; no external funding was received. Competing interests The author declares no competing interests.

Zenodo (CERN European Organization for Nuclear Research)
Shanghai Jiao Tong University (CN), Shanghai Ninth People's Hospital (CN)
Meta-analysis and systematic reviews
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon