Uncertainty quantification for LiDAR-based maps of ditches and natural streams

This article compares novel and existing uncertainty quantification approaches for semantic segmentation used in remote sensing applications. We compare the probability estimates produced by a neural network with Monte Carlo dropout-based approaches, including predictive entropy and mutual information, and conformal prediction-based approaches, including feature conformal prediction (FCP) and a novel approach based on conformal regression. The chosen task focuses on identifying ditches and natural streams based on LiDAR derived digital elevation models. We found that FCP’s uncertainty estimates aligned best with the neural network’s prediction performance, leading to the lowest Area Under the Sparsification Error curve of 0.09. For finding misclassified instances, the network probability was most suitable, requiring a correction of only 3% of the test instances to achieve a Matthews Correlation Coefficient (MCC) of 0.95. Conformal regression produced the best confident maps, which, at 90% confidence, covered 60% of the area and achieved an MCC of 0.82. • The choice of uncertainty quantification method depends on the application. • Feature Conformal Prediction produces most useful uncertainty maps. • Network probability is most suitable for finding misclassified pixels. • Conformal Regression produces confident maps with the highest recall. • Higher DEM resolution improves classification performance and uncertainty estimates.