FairAD-XAI: Evaluation Framework for Explainable AI Methods in Alzheimer's Disease Detection with Fairness-in-the-loop

Despite significant progress in model developments, evaluating eXplainable Artificial Intelligence (XAI) remains elusive and challenging in Alzheimer's Disease (AD) detection using modalities from low-cost or wearable devices. This paper introduces a fine-grained validation framework named 'FairAD-XAI', which provides a comprehensive assessment through twelve properties of explanations, forming a detailed Likert questionnaire. This framework ensures a thorough evaluation of XAI methods, capturing their fairness aspects and supporting the improvement of how humans assess the reliability and transparency of these methods. Moreover, fairness in XAI evaluation is critical, as users from diverse demographic backgrounds may have different perspectives and perceptions towards the system. These variations can lead to biases in human-grounded evaluations and, subsequently, biased decisions from the AI system when deploying. To mitigate this risk, we installed two fairness metrics tailored to assess and ensure fairness in XAI evaluations, promoting more equitable outcomes. In summary, the proposed 'FairAD-XAI' framework provides a comprehensive tool for evaluating XAI methods and assessing the essential aspect of fairness. This makes it a multifactoral tool for developing unbiased XAI methods for AI-based AD detection tools, ensuring these technologies are both effective and equitable.