Quality Control and Interannotator Agreement in Manual Annotation of Damage in Chinese Paintings and Calligraphy

Manual annotation of damage in images of Chinese paintings and calligraphy involves locating damaged areas, assigning damage categories, and delineating their boundaries. Annotators may disagree over the category or extent of the same feature, affecting both AI training data and documentation made before conservation. Drawing on standards for paper-based cultural heritage and image-annotation practice, this methods paper proposes a workflow comprising common annotation rules, independent annotation by two people, discrepancy detection, expert adjudication, and retention of revision records. Using labelme as an example, it describes how to retain two independent annotation files, check for omissions and incorrect categories, and assess agreement with category agreement, intersection over union (IoU), and the Dice coefficient. Five working labels, four operational tables, and three illustrative figures support implementation. The figures and numerical examples are explanatory and are not measured outcomes from collection objects.