Explainable Feature Selection Based on Feature Importance Analysis

Feature selection plays a significant role in reducing the curse of dimensionality and enhancing the performance of deep learning models. To address the shortcomings of existing feature selection techniques in handling complex correlations between features and their lack of explainability, we propose an explainable feature selection method based on feature importance analysis, called FeatureImp. The method evaluates the contribution of features to model predictions through feature perturbation and uses statistical analysis methods to determine the relationships between different features and the potential associations between features and labels. It further identifies combinations of multiple features that may significantly impact the prediction results, thereby optimizing the feature contribution values by simultaneously perturbing multiple features. During the feature selection process, the optimal feature subset is determined based on the feature contribution values and correlation coefficients. Different from existing feature selection methods, FeatureImp quantifies the contribution of features to the model and is applicable to all types of model architectures. We apply FeatureImp to eight public datasets with varying feature dimensions using four types of deep learning models. The results show that FeatureImp reduces the number of features by an average of 30.57%. When models are trained and tested using the datasets after feature selection, the performance of 81.25 % of the models is improved. In addition, FeatureImp is compared with the existing feature selection techniques, the F-measure is increased by an average of 1.57%, and the MAE is decreased by an average of 5.07 %, which validates the effectiveness of FeatureImp.