Improving the explainability of neural network by downstream post-optimization

Explainable Artificial Intelligence (XAI) has garnered significant attention in recent years, aiming to provide comprehensive and insightful explanations of AI decisions to enhance human understanding. While numerous methods have been developed to provide human-comprehensible explanations for AI model decisions, a critical issue remains unresolved: models are prone to co-occurrence bias, leading them to overemphasize irrelevant background features. To solve this problem, we propose a downstream post-optimization method that effectively mitigates co-occurrence bias while generating comprehensive and interpretable explanations of AI decisions. Specifically, we employ the evolutionary algorithm Particle Swarm Optimization (PSO) to produce diverse particles based on the weights of the fully connected layers from a pre-trained model. These particles are subsequently utilized to generate grayscale images for evaluation, where the selection is based on the harmonic average of DAUC and IAUC, with the highest-scoring particle being chosen. The experimental results demonstrate that our method alleviates the impact of co-occurrence bias by allocating more attention to foreground features while reducing the focus on the background, thereby improving the reliability and interpretability of XAI explanations.