A dual-branch Swin Transformer fusion method for fine-grained classification of ancient Chinese jewelry

Abstract Intelligent recognition of the types and materials of ancient Chinese jewelry is essential for digital archiving and cross-collection cataloging. However, high inter-class similarity, natural aging and display-case glare, and long-tailed sample distributions hinder existing methods from jointly capturing shallow texture and deep semantics. We propose a dual-branch fusion classification method based on Swin Transformer. Coordinate attention is embedded in the first two backbone stages to enhance slender jewelry responses and suppress background interference. A dual-branch fusion head exploits shallow color-texture and deep shape semantics. Class-frequency-weighted Focal Loss and a masked multi-task strategy leverage fully and weakly annotated samples. On a self-built dataset, the model achieved 85.00% Top-1 accuracy and 82.45% macro-F1 for jewelry type, and 82.22% Top-1 accuracy and 73.51% macro-F1 for material. It outperformed ResNet-50 and ConvNeXt-T. Ablation experiments validated coordinate attention, dual-branch fusion, and Focal Loss. t-SNE and Grad-CAM showed improved subject focus and material separability. This offers a lightweight path for fine-grained multi-attribute cultural relic recognition and digital heritage preservation.