Comparison and Interpretability Analysis of Deep Learning Models for Classifying the Manufacturing Process of Pigments Used in Cultural Heritage Conservation

This study investigates the classification of pigment-manufacturing processes using deep learning to identify the optimal model for cultural property preservation science. Four convolutional neural networks (CNNs) (i.e., AlexNet, GoogLeNet, ResNet, and VGG) and one vision transformer (ViT) were compared on micrograph datasets of various pigments. Classification performance indicators, receiver-operating characteristic curves, precision–recall curves, and interpretability served as the primary evaluation measures. The CNNs achieved accuracies of 97–99%, while the ViT reached 100%, emerging as the best-performing model. These findings indicate that the ViT has potential for recognizing complex patterns and correctly processing data. However, interpretability using guided backpropagation approaches revealed limitations in the ViT ability to generate class activation maps, making it challenging to understand its internal behavior through this technique. Conversely, CNNs provided more detailed interpretations, offering valuable insights into the learned feature maps and hierarchical data processing. Despite its interpretability challenges, the ViT outperformed the CNNs across all evaluation metrics. This study underscores the potential of deep learning in classifying pigment manufacturing processes and contributes to cultural property conservation science by strengthening its scientific foundation for the conservation and restoration of historical artifacts.