Multi-Parametrized Cross Validation Method based on Order, Size, Weight, and Missing Data Robustness: A New Cross Validation Model MP-OSW-CV

Unless test data is given, the performance of a classifier is calculated with the help of cross validation methods. The existing models focus on different parameters like number of folds, balanced class distribution, test data size etc. However, these models can not overcome bias-variance tradeoff. In this paper, we propose the multi-parametrized cross validation method based on order of an instance, test fold size, weight, and missing data robustness (MP-OSW-CV). This method is composed of four parameters: order, size, weight, and missing data. Firstly, it divides dataset into different parts concerning data indexes and chooses randomly equal number of samples from each part instead of selecting random samples from the whole dataset. Secondly, the test fold size is varied. The accuracy results generated by different test sizes is reflected to the overall performance either with same weights or two different types of inversely proportionally calculated weights. Finally, train size is determined by the last parameter after creating the test fold if missing data robustness is to be analyzed. The proposed method is compared to conventional methods with some datasets from UCI ML Repository. MP-OSW-CV generated more representative data splits, leading to more dependable model assessments.