Semantic Segmentation of Persons in Point Clouds from Photon Counting Lidar

Segmentation of persons at large distances is desired for surveillance and reconnaissance in security and defense applications. Today lidar sensors can provide high-resolution three dimensional (3D) point clouds of scenes at kilometer ranges which can be used for this task. In this work, we evaluate a method for deep learning semantic segmentation of persons in lidar point clouds for long-range surveillance and reconnaissance applications. The method is applied to point clouds from a time-correlated single-photon counting Iidar at ranges from a few hundred meters to around one kilometer in open and forested terrain, which is a new application for person detection. The network utilizes fusion of voxel-based features and point-based features. A slice-based approach is applied, where smaller parts of larger scans are processed separately, which enables near real-time segmentation. The method is trained and validated on annotated data from fifteen different outdoor scenes from three geographical areas, and evaluated on a separate outdoor scene. In experimental results, the method achieves 84.3% in terms of mean Intersection over Union (mIoU) for the person and background classes on the test set. The results show that the presented approach for person segmentation in photon counting lidar data is suitable for long-range applications such as surveillance and reconnaissance.