Using the Out-of-Bag Model in the Cross-Validation Procedure

In the widely known bagging method (random forest), an out-of-bag estimate is generated, which characterizes the quality of the constructed solution. This paper proposes to transfer the idea of constructing this assessment to the cross-validation procedure, which ultimately comes down to a change in the method for constructing the final solution. The resulting method has a smaller variance component in the corresponding error decomposition. Another advantage is that the final solution uses the same models that were used to evaluate the quality during the cross-validation process. This can be particularly significant when the classification method uses significant randomization.