A Commensurate Univariate Variable Ranking Method for Classification

To apply a variable ranking method for feature selection in classification, the notion of commensurateness is necessitated by the presence of different types of independent variables in a dataset. A commensurate ranking method is one that produces consistent and comparable ranking results among independent variables of different types, such as numeric vs. categorical and discrete vs. continuous. We invent a ranking method named conditional empirical expectation (CEE) and demonstrate it is the most commensurate among several representative ranking methods. Further, it has the highest statistical power as a test of independence when the categorical dependent variable is imbalanced. These properties make CEE uniquely suitable for fast feature selection for any datasets, especially those with high dimensionality of mixed types of variables. Its usage is demonstrated with a case study in facilitating preprocessing for classification.