Complete Patient-Level Leakage Among Traceable Images in a Widely Used Brain Tumor MRI Benchmark

Abstract Reported accuracies on a widely used four-class brain tumor MRI benchmark routinely exceed 98%, but the released split discards patient identity. We content-matched tumor images to their figshare source and recovered identifiers for 4586 images from 233 patients. Of 906 tumor test images, 761 were traceable; every one belonged to a patient represented in training. Separately, 216 of 1311 test images were pixel-identical to training images. Removing duplicates did not reduce accuracy (99.22% to 99.42%), whereas five-fold patient-disjoint cross-validation yielded 95.19% overall and 93.68% on tumor classes. On the same 761 images, accuracy fell from 99.12% when the patient appeared in training to 93.60% when absent, increasing the error rate 7.3-fold. A matched random-fold control attributed 2.73 of the 4.03 percentage-point gap to patient grouping rather than the change in evaluation design; the class without patient identifiers changed negligibly despite relaxed proxy grouping. These results show that the released split measures recognition of additional slices from previously seen patients, not generalization to unseen patients. We release the recovered patient map, fixed folds, predictions, and analysis code.