A Simulation Study on Regression Tree Split Criterion using Non-parametric Test Statistics

Recently, algorithm-based machine learning methodologies using large-scale big data have prominent, and decision trees are one of the methodologies. In particular, many studies on ensemble methodologies based on decision trees such as Random Forest and XGBoost have been presented, and these methodologies show excellent performance in large data sets but are not suitable for small samples. For small-sized data, re-sampling methodologies do not effectively provide information about the underlying distribution. In various real-world problems, there are many situations with small samples, but there are not many efficient decision tree methods for them. In this study, we examined the regression trees for small-sized data. In particular, we applied statistical non-parametric test statistics to explore the true split variables in the construction of regression trees, considering test statistics such as Lepage's test statistic, Cucconi's test statistic, and PG's test statistic. As a result of the comparison through simulation studies, decision trees using non-parametric location-scale tests show good performance when dealing with small-sized data.