Source recovery does not guarantee inferential closure: Three lessons for auditing clinical AI claims
Methodological commentary; preprint, not peer reviewed. Background Recovering missing information from a clinical trial report may answer an audit question without supporting every inference drawn from the result. Source recovery, closure of the question as originally specified and evidentiary sufficiency for another use need to be distinguished. Main body This Comment uses three historical source-completion episodes from 18 purposefully assembled development archives. In a childhood-asthma trial, recovered group headers resolved the binding of counts to randomised groups, while eligible and observable participation denominators remained unclear. Programme entry did not directly measure access barriers, but this limitation did not refute a conditional explanation about possible access improvements. In a hypotension trial, table footnotes identified the estimator for a reported difference; this estimate should not be replaced by subtraction of the displayed medians. Repeated-alert aggregation and patient-level interpretation remained unresolved. In a brachytherapy-planning trial, the published layout restored readable symbols but exposed conflicting P-value directions. Reported elapsed planning times covered a fixed phase and did not identify active labour or whole-team savings. Disagreement about closure of the original reading question remains unresolved. The discussion addresses ordinary critical appraisal, moving the goalposts, excessive withholding and documentation burden. The cases are illustrative, not a representative survey, and do not validate an audit method or estimate its causal effect. Conclusions An audit can retain a supported observation, state which question a source repair answers and specify the limitation affecting another intended inference. Reporting these judgments separately avoids treating an apparently complete record as sufficient evidence for a clinical conclusion. AI assistance and human contribution OpenAI Codex agents were used for documentary synthesis of archived evidence, drafting, revision, source navigation, editorial checks, translation, language editing, journal-format adaptation and document preparation. The author directed the project, substantively reviewed the principal arguments, key supporting evidence and limitations, approved the final manuscript and takes responsibility for its content. The source-recovery episodes are illustrative historical reconstructions, not a representative survey, a new method experiment or a validation of clinical accuracy. Funding This work was self-funded; no external funding was received. Competing interests The author declares no competing interests.
Authors
- Chuan Yin (ORCID: https://orcid.org/0009-0005-2830-8167)
Institutions
- Shanghai Jiao Tong University (CN)
- Shanghai Ninth People's Hospital (CN)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-21
- DOI
- https://doi.org/10.5281/zenodo.24410914
- Primary Topic
- Meta-analysis and systematic reviews
- Type
- preprint