Class-Focused Variable Importance in Random Forests for Multi-Class Outcomes

Abstract In multi-class prediction tasks with interpretative goals, covariates that help distinguish individual classes, termed “class-related covariates,” can be of particular interest. Conventional variable importance measures (VIMs) from random forests, such as permutation and Gini importance, rank covariates by overall predictive contribution and thus also assign high importance to covariates that differentiate between groups of classes. We propose a novel VIM, the class-focused VIM, which ranks covariates by their ability to distinguish individual outcome classes. It evaluates covariates using hypothetical multi-way class-based partitions at each node, without altering tree construction. As a complement, we introduce the discriminatory VIM, which measures general covariate influence based on the actual node splits. Simulations show that, unlike conventional VIMs, the class-focused VIM specifically ranks class-related covariates high. Real data examples illustrate how both suggested VIMs behave on real datasets and how their results can be interpreted.