A damage detection network for ancient murals via multi-scale boundary and region feature fusion

Ancient murals are precious cultural heritages. Due to the influence of environmental and human factors, many murals have been damaged to varying degrees. It is an urgent problem for field experts to effectively protect and restore these murals’ original appearance. However, most existing mural restoration works require an automatic and accurate detection technique of the damaged mural regions. In this paper, we propose a damage detection network for ancient murals based on multi-scale boundary and region feature fusion (MBRF). Our proposed network contains two main sub-networks: global information generator (GIG) and detail information generator (DIG). These two sub-networks are, respectively, used for global feature integration and detail feature refinement. In the global feature integration stage, GIG can predict the coarse contents for the damaged mural regions and generate a global mask map. In the feature refinement stage, DIG utilizes the coarse contents of the global mask map to enhance the detailed features and generate the refined feature maps. We design a global decoder that can efficiently fuse multi-level semantic information. Moreover, we introduce a reverse attention (RA) module to capture boundary details. We also propose a bilateral guided fusion (BGF) module to extract and integrate boundary and region features. We conduct experiments on the Mogao Grottoes murals of Dunhuang and the ethnic minority murals in Yunnan. Experimental results show that our proposed network can accurately detect the damaged regions in ancient murals, and outperforms existing detection approaches in terms of subjective visual quality and objective evaluation metrics. The Dice coefficients on the two datasets have reached 0.9173 and 0.7633, respectively, both significantly higher than other comparison approaches.