Virtual restoration of ancient tomb murals based on hyperspectral imaging

Abstract The virtual restoration of historic murals holds immense importance in the realm of cultural heritage preservation. Currently, there are three primary technical issues. First and foremost, it is imperative to delineate the precise location where the mural necessitates restoration. Second, the original color of the mural has changed over time, resulting in a difference from its current appearance. Then, while the method utilizing convolutional neural networks is effective in restoring small defaced areas of murals, its effectiveness significantly diminishes when applied to larger areas. The primary objectives of this paper are as follows: (1) To determine the large and small areas to be restored, the authors employ hyperspectral super-pixel segmentation and support vector machine-Markov random field (SVM-MRF) classification. (2) The authors transform the hyperspectral mural images into more realistic and accurate red-green-blue (RGB) images using the Commission Internationale de l’Eclairage (CIE) standard colorimetric system. (3) The authors restored the images respectively using convolutional neural network and matching image block-based approaches depending on the size of the areas to be mended. The proposed method has enhanced the image quality assessment (IQA) in terms of both color quality and restoration effects. In contrast to the pseudo-color fusion method, the color optimization algorithm described in this research enhances the multi-scale image quality (MUSIQ) by 8.42%. The suggested technique enhances MUSIQ by 2.41% when compared to the convolutional neural network-based image inpainting algorithm.