Automated Segmentation of Stone and Mortar in Heritage Structures: A Case Study on the Old Town Bridge Tower in Prague

Abstract. Accurate digital documentation of heritage structures is vital for conservation, restoration, and structural analysis. Traditional methods for analyzing masonry are time-consuming and subjective. This study proposes a deep learning-based approach using a U-Net convolutional neural network to automatically segment stone and mortar in heritage masonry, trained on high-resolution imagery of Prague’s Old Town Bridge Tower. Unlike prior studies focused on distinct brick structures, our dataset presents a greater challenge due to the similar textures of stone and mortar. Data was collected using a DJI M300 drone with a P1 camera and an RTC360 laser scanner, capturing the entire tower and its interior. The resulting 3D reconstructions and orthophotos, with a 1 mm ground sampling distance, enabled precise manual segmentation of all stones, excluding non-masonry features. After splitting the available manually annotated data, U-Net models with differing parameters were trained on the train set and evaluated on a test set, achieving a class-averaged F1 score of up to 85.58%. The created segmentation maps can be easily converted to finished vector drawings. Results show that deep learning significantly improves segmentation speed and consistency over manual methods. These maps support conservation tasks such as structural monitoring and damage detection. The trained model will aid future documentation of the Charles Bridge, illustrating the potential of AI in advancing scalable, objective digital heritage conservation.