Introducing Feature Importance Consistency (FIC) in XAI Models: A Comprehensive Evaluation using Rank Correlation Coefficients

This article pioneers the concept of Feature Importance Consistency (FIC) in Explainable Artificial Intelligence (XAI), offering a fresh lens to assess and enhance the stability and reliability of feature importance rankings across different models and conditions. FIC, measured through the lens of various rank correlation coefficients including Kendall’s Tau, Pearson, and Spearman, serves as a metric to gauge the agreement in feature importance rankings among different XAI methods and under varied modeling scenarios. Proposing experiments in sentiment analysis and regression modeling, this research theoretically explores the alignment and variations in feature importance rankings provided by well-recognized methods, namely Permutation Feature Importance (PFI), SHapley Additive exPlanations (SHAP), and Local Interpretable Model-agnostic Explanations (LIME). The potential of these experiment’s findings could highlight agreements and differences in feature rankings, emphasizing the vital role of FIC in validating the reliability of identified features and in reducing the risk of inferring incorrect insights from model explanations. Moreover, this work implements a comparative analysis of XAI methods, thereby contributing to the development of more reliable and interpretable AI models across various applications. The introduced FIC concept strengthens the trustworthiness of XAI models and encourages further research into creating standards for evaluating and ensuring feature importance consistency in machine learning models, facilitating a future where AI supports decision-making transparently and interpretability