Variable Selection and Variable Integration for Categorical Dummy Variables in Regression Analysis

Mode selection or variable selection in regression analysis is considered one of the most popular problems to study in empirical research and various theoretical and simulation studies have been conducted. When dealing with categorical data, coding with multiple dummy variables is usually used. However, with this approach, variable selection criteria cannot be systematically applied when some categories are integrated into one category. Most researchers see variables with coefficients near size and integrate them and select variables by minimizing information criteria or other model selection criteria. The underlying reason for the implementation of such cumbersome procedures is related to the lack of systematic variable selection procedures. Finding a way to list all the candidates of the combinations of explanatory variables and their integrated versions remains a major problem. Even if all of them can be listed, estimating the regression models for all the variations of combinations takes too much time, which is another practical problem. To effectively address these issues, in this work, we examined the possibility of utilizing variable selection criteria for ordered categorical data by estimated ordering. We also conducted a simulation study to check the performance of some information criteria.