Hierarchical multi-model ensemble with auxiliary CLIP fusion for fine-grained art style recognition
Fine-grained art style recognition remains difficult because adjacent styles often share subjects, color structures, brushwork, and historical transitions. This study presents a controlled evaluation-and-fusion framework that combines heterogeneous visual classifiers, a low-weight fine-tuned CLIP component, and post-hoc hierarchical diagnostics for ArtBench-10. The resulting Hierarchical Multi-Model Ensemble with Auxiliary CLIP Fusion (HMME-ACF) uses validation-selected probability averaging across five ImageNet-pretrained visual backbones and adds CLIP ViT-B/32 only as a minor auxiliary component. Experiments use 45,000 training images, 5000 validation images, and 10,000 test images under a fixed protocol. The full configuration achieves 86.15% top-1 accuracy, 85.93% Macro-F1, and 86.15% balanced accuracy on the 10-class task. Deterministic post-hoc aggregation yields 92.07%, 89.76%, and 88.03% top-1 accuracy at the period_3, family_4, and family_5 levels, respectively; an explicit hierarchical multitask loss provides only modest coarse-level gains while slightly reducing fine-grained accuracy. The auxiliary CLIP component improves top-1 accuracy by 0.43 percentage points and mainly benefits the most confusable Western style pairs. Accuracy-efficiency analysis shows that the full 84-pass configuration is best suited to offline cataloguing, whereas a five-model no-TTA variant retains 84.27% accuracy at 80 ms per image. These findings position HMME-ACF as a controlled ensemble and diagnostic framework rather than a new hierarchical learning architecture.
Authors
- Haiyin Wang (ORCID: https://orcid.org/0000-0001-9560-0460)
- Zhonglin Wang (ORCID: https://orcid.org/0000-0001-6355-9651)
- Ziqi Dong
- Qing Zhang
Institutions
- Silla University (KR)
- Yancheng Teachers University (CN)
Publication Details
- Journal
- Scientific Reports
- Published
- 2026-08-24
- DOI
- https://doi.org/10.1038/s41598-026-66795-5
- Primary Topic
- Aesthetic Perception and Analysis
- Type
- article
- Field-Weighted Citation Impact
- 0.00