Scalable Bayesian Nonparametric Ensemble (BNE) for Spatio-temporal Air Pollution Predictions using High-Performance Computing

Air pollution is the second leading cause of mortality globally and is linked to numerous adverse health outcomes. Accurate estimation of air pollutant concentrations is critical for air pollution health studies. These estimates are often derived from a range of methods, including machine learning, statistical models, and chemical transport models (CTMs), but the inherent challenges of complex spatio-temporal air pollutant data pose obstacles in modeling. Although many prediction models have been developed by different research groups, their performance varies widely across space and time. Moreover, most models do not provide estimates of the spatio-temporal uncertainty in predictions, especially regarding variation across models. To address these gaps, we developed the Bayesian Nonparametric Ensemble (BNE), a machine learning algorithm that provides a structured approach for spatio-temporal ensemble prediction while incorporating uncertainty and inter-model disagreement. However, BNE is computationally intensive, particularly for predictions when applied at fine spatio-temporal resolutions for health studies. This, along with the need for regular model updates to produce predictions for recent years, requires high-performance computing (HPC) resources. In this paper, we focus on the use of HPC resources—specifically National Science Foundation-funded ACCESS HPC resources—to implement BNE for large-scale daily air pollution prediction at 1 km<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> resolution across the continental U.S. from 2010 to 2016. We discuss computational challenges, including memory, data storage, and runtime limitations, and highlight parallel computing strategies. This work presents a scalable framework for air pollution prediction with spatio-temporal uncertainty, supporting advances in environmental health and policy.