Detection of maintenance issues from UAV images of archaeological sites: A yolo-based tool

Abstract Protecting cultural heritage (CH) is a strategic activity for all countries, like Italy, which has many ancient properties that can be degraded and damaged over time. Archaeological sites are critical CH assets for Italy, and their management and protection are crucial. Visitors can intentionally or unintentionally damage archaeological sites, while natural events like rain, wind, sun, and weeds can degrade or damage such CH assets. In such a context, modern technologies can effectively support monitoring activities. This paper presents the design of a framework for acquiring aerial images and their analysis, supporting operators on a site with detailed maintenance suggestions and information, and allowing the launch of new precision surveys to investigate identified issues better. We also propose a prototype tool for automatically detecting maintenance issues in an archaeological site based on AI models applied to aerial orthophotos of the site. The case study taken in the exam is related to the archaeological site of Pompeii, which provides high-definition orthophotos of its artistic resources using aerial drones. A prototype tool is proposed to discover such maintenance issues in such images rapidly, present them straightforwardly to support human operators’ decision-making and understand which site zone needs more attention. The maintenance issues to identify fall into four classes: weedy vegetation, damaged conduits, damaged structures, and broken tiles. In the experimental phase, a custom dataset was used to train and evaluate various versions of Yolo model. The best performance has been obtained through the YoloV5l detector, with a F1 score of 0.482, 0.427 for mAP0.5 and 0.264 for mAP0.5–0.95 on cross-validation, a 0.502 F1-score, 0.482 mAP50 and 0.279 mAP50-95 on the test set. The model has a FPS capacity of 54.945 frame/sec. The tool has proven good efficiency being capable of scanning and analyse an entire orthophoto of about 10 GBs in a few minutes.