Two-stage mural image restoration using an edge-constrained attention mechanism

The current mainstream image restoration methods have difficulty fully learning the structure and color information of murals in mural image restoration tasks due to the limited size of the available datasets, resulting in problems such as structural loss and texture errors. This study proposes a two-stage mural restoration network based on an edge-constrained attention mechanism. This paper introduces additional sketches as inputs during the coarse restoration phase and incorporates a local edge loss function to enable the network to generate corresponding structural information based on the sketches. In the fine restoration phase, the calculation for the similarity between missing areas and known areas is optimized to enhance the consistency of the restoration results with the texture of the known areas. Furthermore, a structure-guided attention propagation block is introduced after adopting the attention mechanism. This block selectively integrates surrounding contextual information to update the attention score map, thereby enhancing the coherence and plausibility of the generated textures. The experimental results show that the proposed method outperforms the current mainstream restoration methods according to various assessment indices. The proposed method generates high-quality structural information according to user guidance information, and the repaired texture is highly visually consistent with that of the original mural, with few noticeable deviations. This study provides a new approach for mural restoration, which may positively impact cultural heritage protection and artistic restoration applications.