Constructing Ensembles: A Diversity-Driven Approach with Correlation and Q-Statistics

Machine learning models often encounter limitations such as overfitting and sensitivity to high-dimensional data, which can compromise their performance. Ensemble learning mitigates these issues by combining multiple models to enhance overall accuracy and robustness. The effectiveness of an ensemble largely depends on the diversity and accuracy of its constituent models-diversity ensures that individual models make different errors, while accuracy ensures each model positively contributes to the ensemble's performance. This paper proposes constructing ensemble methods by selecting individual models from a pool of 14 classifiers based on two diversity metrics: correlation and Q-statistics, and adopting four strategies to construct the proposed en-sembles. To empirically validate our proposed approach, four widely available datasets have been used, and 10-fold cross-validation was conducted using seven performance indicators. The results suggest that the diversity of classifiers is highly dataset-dependent, with neither correlation nor Q-statistics consistently leading to the most accurate ensembles across all datasets. Balancing diversity and performance accuracy is important to build successful ensembles rather than relying only on diversity metrics.