LiDAR and hyperspectral remote sensing image fusion using extended extrema morphological profiles for semantic segmentation

Semantic segmentation of natural scenes is an important basic work of remote sensing information processing. LiDAR can collect the elevation information of the scene, and hyperspectral remote sensing images can collect the unique spectral information of the scene objects. Making full use of the data collected by different kinds of sensors can effectively improve the accuracy of semantic segmentation. To this end, this paper proposes a novel semantic segmentation method by fusing the extended extrema morphological profiles (EEMPs) of LiDAR and hyperspectral image (HSI). Firstly, principal component analysis (PCA) is used as the feature extractor to construct the feature maps by extracting the first informative feature from LiDAR and HSI data. Secondly, the extrema morphological profiles (EMPs) are used to extract the spatial structure feature from the informative feature maps to construct the EEMPs. Thirdly, in order to synthesize the elevation information of LiDAR and the spectral information of HSI in the scene, a feature-level image fusion method of LiDAR and HSI based on total variation model is adopted. Finally, support vector machine (SVM) is utilized to obtain accurate semantic segmentation from the fused EEMPs. Four metrics, i.e., class accuracy (CA), overall accuracy (OA), average accuracy (AA), and Kappa coefficient, are used to quantitatively measure the evaluate the semantic segmentation results. The high semantic segmentation accuracy demonstrate that EEMPs can efficiently extract the complementary information from LiDAR and HSI data.