Reducing Meta-Level Data in Stacking Ensembles for Classification

Stacking as an ensemble framework relies on meta-learning for combining the outputs of multiple base classifiers. What exactly the outputs represent and how they are used for meta-learning is an important aspect of stacking. Commonly, the prediction probabilities produced by the base classifiers at the class level are used for training the meta-classifier. The dimensionality of this meta-level data is, therefore, not negligible. Reducing it would result in the need for a less complex meta-classifier, and would thereby simplify the process of meta-learning. This paper presents a straightforward approach for aggregating the prediction probabilities of the base classifiers. By calculating the mean and standard deviation at the class level, a substantial reduction in the dimensionality of the meta-level data is achieved. The experimental analysis, conducted on multiple diverse datasets, suggests that the aggregation preserves sufficient information for meta-learning since better or highly competitive performance was attained with respect to the common stacking ensemble framework (utilising the full meta-level data).