Towards reliable land cover mapping under domain shift: An overview and comprehensive comparative study on uncertainty estimation

An increasing number of land cover products have been generated from remote sensing imagery by deep learning based semantic segmentation models, attributable to their substantial advancements in performance relative to traditional machine learning techniques. However, due to the spatial-temporal-spectral heterogeneity in the complex and diverse remote sensing imageries, the occurrence of discrepancies between the distribution of the test data and the training data, which is also known as domain shift, is common in the application phase of the deep learning model, resulting in a significant number of errors in the model predictions. These errors will introduce inaccuracies and uncertainty to application of the products generated with domain shift. Developing corresponding pixel-wise uncertainty estimation products for these land cover products is one of the promising ways to alleviating the above challenge. However, there is a scarcity of relevant research and products in the field of deep learning based land cover mapping. This paper aims to fill this research gap by providing an overview and comprehensive comparative study on uncertainty estimation for deep learning based land cover mapping under domain shift. This overview not only summarizes the concepts, methods and evaluations on uncertainty estimation, but also elaborates on its current application status in land cover mapping and values in addressing challenges from domain shift. Moreover, we provide a comparative study of ten practical uncertainty estimation methods by quantitatively assessing their performance on four tailor-made land cover datasets related to four common types of domain shift. Consequently, many valuable insights for research and application of uncertainty estimation are revealed. For example, the learning based method which has not been previously applied in the field of remote sensing demonstrates strong performance across most types of domain gap, expect for spectral gap, while the commonly utilized Monte Carlo Dropout method exhibits only average performance. We hope this work can promote the development of uncertainty estimation products of land cover classification, as well as facilitate the progression of reliable mapping techniques under domain shift. • Review on uncertainty estimation for land cover mapping using deep learning model. • Comparative study of 10 uncertainty estimation methods using 4 datasets. • 4 domain shifts due to spatial-temporal-spectral heterogeneity are considered. • Both direct and indirect evaluation are applied in assessment. • The learning based method exhibits the best performance.