Evaluating Explainability in XAI: A Case Study of Bug Assignment and Developer Allocation

Explainable Artificial Intelligence (XAI) has gained significant importance in software engineering, especially in tasks like bug assignment and developer allocation, which require trust and transparency. However, what constitutes a satisfactory explanation is difficult to determine because user satisfaction may be subjective and stakeholder expectations may differ. This paper provides a quality evaluation framework for explain ability, applied to bug assignment and developer allocation. It further categorizes into multiple dimensions, such as transparency, interpretability, and satisfaction of users, tailored to the needs of software development teams. We propose a framework to evaluate which types of explanations are helpful and assign weights to various criteria with respect to their introduction into the development process. The proposed framework supports assessing whether the XAI system improves decision-making support as intended. We illustrate the application of our framework in real-world scenarios through case studies and empirical analysis, providing guidance on how to maximize trust in XAI systems while also improving efficiency in software development workflows.