PERFEX-I: confidence scores for image classification using decision trees

To be able to use machine learning models in practice, it is important to know when their predictions can be trusted. Confidence estimations can help end users to calibrate their trust, avoiding under- or over-reliance, and to decide when human interference is needed. In our work, we further develop the eXplainable AI (XAI) method PERformance EXplainer (PERFEX), which was originally proposed for tabular datasets. We adapt PERFEX such that it can be used to accurately estimate the image classifier confidence. This was done by applying the method on feature-reduced activation values of the last layer of image classification models. We coin this approach PERFEX-I. We show that PERFEX-I performs on par with existing methods for confidence estimation such as Temperature Scaling and Deep Ensembles. The Expected Calibration Error (ECE) on the ImageNet dataset is reduced from 6.83 to 1.71 for ResNet50 and from 8.84 to 1.44 for Swin-B compared to using the Softmax scores. Additionally, PERFEX-I groups images that may share common reasons for errors, and visual analysis of these groups can reveal patterns of the model’s behavior.