Independent evaluation of Aurora air pollution forecasts beyond benchmark skill

Abstract We evaluate the released Aurora 0.4-degree Air Pollution checkpoint against CAMS, persistence, and hourly background-station observations for 96 initializations spanning 2024. Against CAMS, Aurora reduces mean squared error relative to persistence by 38.1-42.6% for PM2.5 and 49.1-52.3% for PM10. At sampled station times, its mean absolute error is lower than that of the benchmark-selected CAMS fields in all six combinations of three regions and two pollutants; 95% initialization-date bootstrap intervals exclude zero. At +12 h, 6.27% of global PM2.5 and 7.84% of PM10 grid-point values are negative, whereas matched CAMS states are non-negative; subsequent forecast steps use positivity clamping. The evaluated outputs provide no forecast distribution. Perturbing four static source-related inputs produces small, nearly additive regional responses, whose relationship to changes in actual emissions remains untested. Observational agreement, physical constraints, and input sensitivity thus yield distinct assessments of the same forecasts.