SPARK: a universal approach to 3D point cloud segmentation using 2D image segmentation models – cemonstration on traffic objects

The generation of point clouds using LIDAR or photogrammetry is used in various fields of (civil) engineering and allows to capture the geometry of scenes or objects. The categorisation of objects and their segmentation has to be done either manually or using isolated solutions with limited functionality. This paper presents the general purpose application SPARK, which uses 2D object recognition to perform automated 3D object segmentation in point clouds. The point cloud mesh is imported into the Unity game engine and an orthographic camera is flown over the mesh to capture images. 2D object segmentation CNNs are then used to classify and mask the detected objects. This information is projected onto the mesh using raycasts to create a three-dimensional localisation of the classified points in the scene. These points are clustered into bounding boxes and used to localise elements in the original point cloud and to create classified sub-point clouds. The workflow is demonstrated in the SPARK application using the example of capturing road objects with the object segmentation models pre-trained on the ADE20K and Cityscapes datasets. A case study with three point clouds of urban street scenes is performed and the results and applicability are discussed.