Multi-Viewpoint Evaluation of Explanation Quality in X-IDS Using Aggregated and Consensus Metrics

Explainable intrusion detection systems (X-IDS) typically provide a set of explanations for each alert, while provided explanations may not be sufficient for security analysts to make time-critical decisions. Existing evaluation methods only consider such cases with a single set of explanations, as a result, limiting the scope of evaluation. This work expands a set of explanation evaluation metrics, our earlier work, to extend the scope of evaluation covering X-IDS providing multiple sets of explanations. Additional intermediate metrics are proposed to capture characteristics of multiple sets of explanations so the evaluation metrics can be computed for the multiple sets of explanations. The experimental results show the proposed metrics reveal more insights.