An Operations-Based Evaluation of Tropical Cyclone Track and Intensity Forecasts from Artificial Intelligence Weather Prediction Models

Abstract In just the past few years, multiple data-driven Artificial Intelligence Weather Prediction (AIWP) models have been developed. NHC verification procedures are used to evaluate 7-day track and intensity predictions for Northern Hemisphere tropical cyclones (TCs) from May to November 2023. Four open-source AIWP models initialized 4 times per day with GFS initial conditions are considered (FourCastNetv1, FourCastNetv2-small, GraphCast-operational, and Pangu-Weather). The AIWP track forecast errors and detection rates are comparable to those from the best-performing operational forecast models. However, the AIWP intensity forecast errors are larger than those of even the simplest intensity forecasts based on climatology and persistence. The AIWP models almost always reduce the TC intensity resulting in a substantial low bias. An idealized Rankine vortex was used to illustrate how minimizing RMS errors in the AIWP models contributes to this bias. The contribution of the AIWP models to the NHC model consensus was also evaluated. For track, adding the AIWP models improves the consensus by up to 11% at the longer time periods. This represents more than a 5-yr gain in accuracy based on the historical track forecast improvement of about 2% yr −1 . In contrast, the AIWP models did not improve the intensity consensus. A preliminary evaluation of the impact of the AIWP model initial conditions was performed by comparing forecasts initialized with GFS and ECMWF-Integrated Forecasting System (IFS) initial conditions for Northern Hemisphere cases in 2022–23. With the ECMWF-IFS initial conditions, the AIWP track forecasts significantly improved, but there was little improvement in the intensity forecasts. Significance Statement Weather prediction (WP) models based on artificial intelligence (AI) have proliferated over just the past few years. Once trained, these models run very quickly, making them appealing for operational forecasting. This study is one of the first to apply operational verification procedures to multiple open-source AIWP models to determine their applicability for operational forecasting. The track forecast errors are found to be comparable to those from the best operational models. However, the intensity forecasts have no skill relative to the simplest statistical models, due to a substantial low bias. Thus, the tropical cyclone (TC) track forecasts can be used directly as guidance, but postprocessing bias corrections or AIWP model architecture and training data modifications may be needed for useful TC intensity forecasts.