Heritage-aware visual representation learning with layout and motif relation modeling for Nanjing Yun brocade image classification

Intelligent classification of Yun brocade images is of great significance for the digital organization of traditional brocade resources, the preservation of intangible cultural heritage, and intelligent analysis of cultural images. To address the challenges of complex motif structures, subtle category differences, and the difficulty of jointly modeling global layout and local motif relationships in Yun brocade images, this paper constructs a Yun brocade image classification method that integrates layout priors and motif relation modeling. The proposed method adopts the visual Transformer as the basic framework, designs the Brocade Layout Weaver (BLW) module to model Yun brocade craft layouts such as central regions, boundary regions, corner regions, and warp-weft structures, and further designs the Motif Structure Relation Composer (MSRC) module to enhance the model’s representation ability for repeated motifs, symmetric structures, and local compositional relationships through motif anchor extraction and spatial structural relation modeling. This paper constructs a self-built Yun brocade image dataset containing three sub-datasets for temporal classification, variety classification, and motif classification, and conducts comparative verification with multiple mainstream visual models. Experimental results show that the proposed method achieves an Accuracy of \(91.72\%\) , an F1-Score of \(91.20\%\) , and an AUC of \(98.64\%\) on the temporal classification sub-dataset; an Accuracy of \(81.38\%\) , an F1-Score of \(73.03\%\) , and an AUC of \(94.12\%\) on the variety classification sub-dataset; and an Accuracy of \(91.01\%\) , an F1-Score of \(93.64\%\) , and an AUC of \(97.98\%\) on the motif classification sub-dataset, all outperforming the comparison models. Ablation experiments further demonstrate that BLW and MSRC can respectively enhance the model’s representation ability for the global craft layout and local motif structural relationships of Yun brocade, and their joint use can form more discriminative visual representations of Yun brocade images.