Auditable Evaluation Environments for Medical Signal and Imaging Tasks: A Preprint (v0)

Medical-AI benchmarks usually report a number that a third party cannot re-derive. We describe a small framework in which every task ships three inseparable parts: data with known ground truth, a grader that reads only a prediction file and a truth file, and a reproducibility veto that voids the whole submission when the same seed does not reproduce frame-for-frame. Two tasks are implemented — resting-state EEG alpha-peak identification and synthetic OCT layer-boundary localisation. Rule-based baselines score 100/100 on the EEG task and 20% pass-rate on the OCT task; we read the first as a statement about synthetic data rather than about EEG. No clinical claim is made.Code: https://github.com/Even-li316/med-eval-envs (MIT). PDF SHA-256: eb0f082185704e079775fd9c5f110c2a3b0c672fcd859d9d00e62d1863f77316

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-17
DOI
https://doi.org/10.5281/zenodo.24300158
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Auditable Evaluation Environments for Medical Signal and Imaging Tasks: A Preprint (v0)

Li Yafeng
5 links
Zenodo (CERN European Organization for Nuclear Research)
Artificial Intelligence in Healthcare and Education
preprint

Auditable Evaluation Environments for Medical Signal and Imaging Tasks: A Preprint (v0)

Li Yafeng
preprint en

Abstract

Medical-AI benchmarks usually report a number that a third party cannot re-derive. We describe a small framework in which every task ships three inseparable parts: data with known ground truth, a grader that reads only a prediction file and a truth file, and a reproducibility veto that voids the whole submission when the same seed does not reproduce frame-for-frame. Two tasks are implemented — resting-state EEG alpha-peak identification and synthetic OCT layer-boundary localisation. Rule-based baselines score 100/100 on the EEG task and 20% pass-rate on the OCT task; we read the first as a statement about synthetic data rather than about EEG. No clinical claim is made.Code: https://github.com/Even-li316/med-eval-envs (MIT). PDF SHA-256: eb0f082185704e079775fd9c5f110c2a3b0c672fcd859d9d00e62d1863f77316

Zenodo (CERN European Organization for Nuclear Research)
Artificial Intelligence in Healthcare and Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon