From radiology benchmark scores to patient benefit: a sealed exploratory evaluation of Athena, an incomplete stock-model control, and a conditional impact model
AI-assisted exploratory research preprint, not peer reviewed or clinically validated. A public-availability radiology subset contained 210 questions. The live Athena session scored 191/210. On the same 110 completed stock-control questions, Athena scored 97/110 and stock Astra 100/110; the other 100 control questions are missing. This incomplete, non-resource-matched comparison does not demonstrate an Athena advantage or superiority over Harrison models. A post-hoc ranked-key scoring correction, prior exposure, native-image delivery, bundled-question cueing, uncertain exact checkpoint identity and an interrupted paid control are explicitly disclosed. The supplement provides original outcome flags, file hashes and executable hypothetical impact arithmetic. No patient mortality effect is measured. The lives-saved scenarios are unfitted illustrations that include zero benefit and harm, not forecasts. Original radiographs, copyrighted question text, private operational scaffold, personal medical/legal records and credentials are not redistributed. AI assistance, proprietary-method limitations and the author's interest in the system are disclosed.
Authors
- R. E. W. Kho
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-09-09
- DOI
- https://doi.org/10.5281/zenodo.22666681
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- preprint