A Deep Learning Multiview Approach for the Semantic Segmentation of Heritage Building Point Clouds

Digitization powered by computer-aided techniques represents a powerful tool for advancing documentation of heritage structures. Nonetheless, creating as-built models remains a challenging task, particularly when it comes to handling large-scale point cloud data from multiple acquisitions. Efficient strategies are needed to interpret this complex data effectively. In response, this study proposes an original point cloud semantic segmentation method for heritage building point clouds, leveraging a multi-view deep learning framework. The process involves two key stages: first, a convolutional neural network is employed to retrieve semantic features from a series of images obtained through a photogrammetric survey. Then, using intrinsic and extrinsic camera parameters, the semantic data is projected onto the 3D photogrammetric reconstruction. To assess this approach, a set of tests was carried out using an image-point dataset comprising five scenes. The results of the initial tests are promising, although the still limited amount of training data show some limitations in terms of generalization and capability. Nevertheless, the procedure stands out for its remarkable ease of use and utility in processing photogrammetric point clouds. The preliminary results suggest that the procedure could bring significant advantages to the segmentation of point clouds of historic buildings.