AI-Enabled conditional nonlinear optimal perturbation enhances ensemble prediction of extreme El Niño events

Data-driven deep learning (DL) models often underestimate the intensity of extreme weather and climate events due to the scarcity of extreme samples in training datasets and the smoothing effects of gradient-based optimization. While ensemble prediction methods based on initial condition (IC) perturbations in traditional numerical models have improved extreme event predictions, they often fail in DL frameworks. This is primarily due to limited error growth characteristics and the implicit regularization in DL models, which dampens the amplification of IC perturbations. To overcome this limitation, we introduce a novel IC perturbation scheme based on orthogonal conditional nonlinear optimal perturbation (O-CNOP), integrated into a DL-based ensemble prediction system. The O-CNOP-derived perturbations are obtained through an iterative selection and optimization process, beginning with candidate samples from model simulations under uniform energy constraints. Perturbations are then selected to maximize forecast error growth, guided by ensemble averaging and convergence criteria. We evaluate this method based on four major El Niño events (1982/83, 1997/98, 2015/16, and 2023/24). Results show significant improvements in DL model predictions when initialized in spring, with over a 30% reduction in prediction error for Niño3.4 sea surface temperature anomalies. This AI-enabled O-CNOP framework offers a robust and generalizable approach to ensemble predicting, potentially improving the prediction skill of DL-based weather and climate models for extreme events.