Clever Hans effect found in a widely used Alzheimer's Disease MRI dataset

Motivation: While recent studies show high accuracy in the classification of Alzheimer's disease using deep neural networks, the influence of T1-weighted gray-white texture and preprocessing have not been investigated. Goal(s): To systematically identify the features learned by the deep neural network for model validation. Approach: Using T1-weighted images we separated Alzheimer's patients (n=201) from healthy controls (n=159) by using a deep neural network and systematically compared the performances using McNemar tests and the learned features using Layer-wise Relevance Propagation (LRP). Results: Binarized MR images yielded comparable classification performance, demonstrating a neglectable impact of T1-weighted gray-white matter texture. Learned features are similar. Impact: The identification of biases learned by deep neural networks for disease classification using only structural MR images enables validation of the models and improves reliability.