Auditable Evaluation Environments for Medical Signal and Imaging Tasks: A Preprint (v0)
Medical-AI benchmarks usually report a number that a third party cannot re-derive. We describe a small framework in which every task ships three inseparable parts: data with known ground truth, a grader that reads only a prediction file and a truth file, and a reproducibility veto that voids the whole submission when the same seed does not reproduce frame-for-frame. Two tasks are implemented — resting-state EEG alpha-peak identification and synthetic OCT layer-boundary localisation. Rule-based baselines score 100/100 on the EEG task and 20% pass-rate on the OCT task; we read the first as a statement about synthetic data rather than about EEG. No clinical claim is made.Code: https://github.com/Even-li316/med-eval-envs (MIT). PDF SHA-256: eb0f082185704e079775fd9c5f110c2a3b0c672fcd859d9d00e62d1863f77316
Authors
- Li Yafeng
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-17
- DOI
- https://doi.org/10.5281/zenodo.24300158
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- preprint