Enhancing Image Classification with Ensemble Deep Learning Through Deep Feature Concatenation

Ensemble deep learning represents a powerful methodology that combines multiple deep learning models to improve overall performance. By aggregating predictions or features from these models, ensemble methods achieve superior accuracy, robustness, and generalization compared to individual models. In this paper, we introduce a novel approach to enhancing classification performance through ensemble deep learning techniques. Our method leverages the final extracted features from several pre-trained convolutional neural networks (CNNs). Each CNN independently generates a feature vector from the same input image, and these vectors are concatenated to integrate the diverse outputs of each model. The resulting concatenated feature vector is subsequently fed into a classifier, specifically fully connected layers, which enhances the robustness and generalization capabilities of the classification task. This approach demonstrates that concatenating final extracted features from different models can significantly elevate classification accuracy. Experimental results on the Cats vs Dogs dataset reveal notable improvements in classification accuracy over single-model approaches. Specifically, VGG16 and VGG19 achieved accuracies of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$91.71 \%$</tex> and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$90.18 \%$</tex>, respectively. The ensemble of outputs from VGG16 and VGG19 resulted in a substantial accuracy boost, reaching <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\text{93.75 \%}$</tex>. Inception-V3 exhibited a higher accuracy of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\text{97.75 \%}$</tex>, and when combined with VGG19 and VGG16, it achieved accuracies of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\text{97.96 \%}$</tex> and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\text{98.12 \%}$</tex>, respectively. The ensemble comprising InceptionV3, VGG16, and VGG19 yielded the highest performance, with an accuracy of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\text{98.48 \%}$</tex> and a low error rate of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$4.50 \%$</tex>. Furthermore, EfficientNetB0 recorded an accuracy of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\text{96.67 \%}$</tex>, which improved to <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\text{98.32 \%}$</tex> when integrated with Inception-V3. Notably, MobileNet achieved an accuracy of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$98.00 \%$</tex> with an error rate of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$0.09 \%$</tex>, while the combination of MobileNet and VGG16 further enhanced performance, resulting in an accuracy of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\text{99.00 \%}$</tex> and an error rate of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\text{0.18 \%}$</tex>.