Hierarchical multi-model ensemble with auxiliary CLIP fusion for fine-grained art style recognition

Fine-grained art style recognition remains difficult because adjacent styles often share subjects, color structures, brushwork, and historical transitions. This study presents a controlled evaluation-and-fusion framework that combines heterogeneous visual classifiers, a low-weight fine-tuned CLIP component, and post-hoc hierarchical diagnostics for ArtBench-10. The resulting Hierarchical Multi-Model Ensemble with Auxiliary CLIP Fusion (HMME-ACF) uses validation-selected probability averaging across five ImageNet-pretrained visual backbones and adds CLIP ViT-B/32 only as a minor auxiliary component. Experiments use 45,000 training images, 5000 validation images, and 10,000 test images under a fixed protocol. The full configuration achieves 86.15% top-1 accuracy, 85.93% Macro-F1, and 86.15% balanced accuracy on the 10-class task. Deterministic post-hoc aggregation yields 92.07%, 89.76%, and 88.03% top-1 accuracy at the period_3, family_4, and family_5 levels, respectively; an explicit hierarchical multitask loss provides only modest coarse-level gains while slightly reducing fine-grained accuracy. The auxiliary CLIP component improves top-1 accuracy by 0.43 percentage points and mainly benefits the most confusable Western style pairs. Accuracy-efficiency analysis shows that the full 84-pass configuration is best suited to offline cataloguing, whereas a five-model no-TTA variant retains 84.27% accuracy at 80 ms per image. These findings position HMME-ACF as a controlled ensemble and diagnostic framework rather than a new hierarchical learning architecture.

Authors

Institutions

Publication Details

Journal
Scientific Reports
Published
2026-08-24
DOI
https://doi.org/10.1038/s41598-026-66795-5
Primary Topic
Aesthetic Perception and Analysis
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Hierarchical multi-model ensemble with auxiliary CLIP fusion for fine-grained art style recognition

Haiyin Wang, Zhonglin Wang, Ziqi Dong, Qing Zhang
Scientific Reports
Aesthetic Perception and Analysis
article

Hierarchical multi-model ensemble with auxiliary CLIP fusion for fine-grained art style recognition

Haiyin Wang, Zhonglin Wang, Ziqi Dong, Qing Zhang
article en

Abstract

Fine-grained art style recognition remains difficult because adjacent styles often share subjects, color structures, brushwork, and historical transitions. This study presents a controlled evaluation-and-fusion framework that combines heterogeneous visual classifiers, a low-weight fine-tuned CLIP component, and post-hoc hierarchical diagnostics for ArtBench-10. The resulting Hierarchical Multi-Model Ensemble with Auxiliary CLIP Fusion (HMME-ACF) uses validation-selected probability averaging across five ImageNet-pretrained visual backbones and adds CLIP ViT-B/32 only as a minor auxiliary component. Experiments use 45,000 training images, 5000 validation images, and 10,000 test images under a fixed protocol. The full configuration achieves 86.15% top-1 accuracy, 85.93% Macro-F1, and 86.15% balanced accuracy on the 10-class task. Deterministic post-hoc aggregation yields 92.07%, 89.76%, and 88.03% top-1 accuracy at the period_3, family_4, and family_5 levels, respectively; an explicit hierarchical multitask loss provides only modest coarse-level gains while slightly reducing fine-grained accuracy. The auxiliary CLIP component improves top-1 accuracy by 0.43 percentage points and mainly benefits the most confusable Western style pairs. Accuracy-efficiency analysis shows that the full 84-pass configuration is best suited to offline cataloguing, whereas a five-model no-TTA variant retains 84.27% accuracy at 80 ms per image. These findings position HMME-ACF as a controlled ensemble and diagnostic framework rather than a new hierarchical learning architecture.

Scientific Reports
Silla University (KR), Yancheng Teachers University (CN)
Openalex Percentile: Top 9%
Aesthetic Perception and Analysis
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.