Uncertainty quantification in geospatial AI/ML applications: methods, metrics, and open-source support with an air quality use case

The adoption of AI/ML models in geospatial applications faces challenges related to trustworthiness and reliability due to uncertainties in model processing and datasets. Such uncertainties are exacerbated by the complexity of Earth’s systems and constraints, such as data quality and measurement accuracy. Despite growing interest, the field lacks a systematic comparison of Uncertainty Quantification (UQ) methods tailored to the unique challenges of geospatial modelling, particularly in real-time settings. These challenges should be addressed by integrating UQ into geospatial workflows, enabling stakeholders and users to understand model uncertainties and make decisions with confidence. This paper addresses these gaps by systematically evaluating three popular UQ methods, focusing on their suitability for geospatial applications and their ability to address predictive uncertainties. Additionally, we analyzed various UQ metrics, highlighting their role in assessing predictive capacity and calibration. However, improvements are needed to establish unified frameworks and streamline integration for real-time and large-scale applications. A case study on PM2.5 calibration demonstrated the practical utility of UQ in enhancing air quality (AQ) data reliability. Results highlight that Deep Ensembles (DE) in TensorFlow achieved the strongest performance, followed by BNN in TensorFlow, which offered dependable calibration and accuracy for the case study. MCD in TensorFlow came next, delivering solid performance but with less adaptable uncertainty at extreme PM2.5 values. MCD in PyTorch performed the poorest, with lower accuracy and unreliable calibration. These results demonstrate that TensorFlow consistently outperformed PyTorch across methods, underscoring framework-specific differences in implementing UQ. Overall, this study conducted a comprehensive review and experimental study of UQ methods and frameworks, offering critical insights into their theoretical foundations, practical implementation, and performance in geospatial applications using PM2.5 calibration as an example.