Relic fragment matching based on hierarchies of features

Computer-aided cultural heritage restoration technology has made significant progress in recent years. However, existing algorithms usually focus on individual criterion, which might only apply to certain scenarios. To handle various kinds of fragments, expert knowledge is still needed. In this work, we propose an interactive restoration system that combines learning models and expert knowledge in an intuitive manner. We model the relics at three hierarchies from global to local scales: fragments, borderline curves, and segments. Our learning model consists of an attention-based encoder to represent curve features for local segment matching, and a graph neural network for fragment-level link prediction. The learning-based matching results are visualized for user exploration and editing. We evaluate our system quantitatively and qualitatively using fragments generated by scanned relics from Guangdong Museums. The networks are assessed through curve-based segment matching and graph in varying conditions. The interactive interface is examined through usage scenarios and metrics.