Robust Multiclass Pneumonia Classification via Multi-Head Attention and Transfer Learning Ensemble

Pneumonia is an acute respiratory infection caused by pathogens such as bacteria or viruses, and accurate early diagnosis is critical for reducing mortality. Chest X-ray (CXR) imaging serves as a conventional diagnostic tool. However, radiographic features of pneumonia often overlap with those of other pulmonary diseases and are subject to inter-observer variability. Traditional Convolutional Neural Network (CNN) models tend to capture redundant information during feature extraction, and single pre-trained models often exhibit limited generalization in multiclass classification tasks. This study proposes a multi-model ensemble learning framework based on multi-head attention mechanism. Firstly, the three pre-trained backbones—DenseNet-121, ResNet-50, and VGG-19—were fine-tuned through transfer learning by replacing their classification heads, adapting pooling layers, and optimizing the fully connected layers. Secondly, feature maps extracted from these tuned backbones were concatenated and fused using a multi-head attention mechanism; the fused representation was then refined by two consecutive multi-head attention layers and finally passed to a fully connected classifier to produce the ensemble prediction. Three task sets were constructed from a public Kaggle dataset: binary classification (normal vs. pneumonia), three-class classification (normal, COVID-19, viral pneumonia), and four-class classification (normal, lung opacity, viral pneumonia, COVID-19), achieving accuracies of 91.67%, 93.79%, and 90.60%, respectively. The results demonstrate that the proposed multi-head attention-based ensemble framework offers significant advantages for pneumonia multiclass classification, particularly by maintaining high recall and robustness in more complex scenarios such as four-class differentiation, indicating its potential as a clinical decision-support tool. Future work will involve expanding the dataset and evaluating the model’s generalizability across additional disease categories.