Class-specific feature selection for enhancing explainability in ensemble classification models

Abstract Feature selection techniques aim at finding a relevant subset of features that perform equally or better than the original set of features at explaining the behavior of data. Typically, features are extracted from feature ranking or subset selection techniques, and the performance is measured by classification or regression tasks. However, while selected features may not have equal importance for the task, they do have equal importance for each class. The fundamental idea of the class-specific concept resides in the understanding that the significance of each feature can vary from one class to another. This contrasts with the traditional class-independent approach, which evaluates the importance of attributes collectively for all classes. For example, in tumor prediction scenarios, each type of tumor (class) may be associated with a distinct subset of relevant features. These features possess significant discriminatory power, enabling the differentiation of one tumor type from others (classes). This class-specific perspective offers a more effective approach to classification tasks by recognizing and leveraging the unique characteristics of each class. A novel deep one-versus-each strategy is introduced, which offers advantages from the point of view of explainability (feature selection) and decomposability (classification). In addition, the class-specific relevance matrix is presented, from which some more sophisticated classification schemes can be derived, such as the three-layer class-specific scheme. These schemes have the great advantage to combine independent classification units (e.g., neural networks) that use a reduced number of features to target each class. The potential for further advancements in this area is wide and will open new horizons for exploring novel research directions in interdisciplinary fields, particularly in complex, multiclass hyperdimensional contexts (e.g., in genomics).