A lightning augmented recurrent nowcasting model based on self-supervised learning and multi-modal fusion method

The heavy class imbalance problem and the multi-source observations fusion remain challenges for lightning nowcasting based on deep learning method. To address the problems, this paper proposes a novel lightning augmented recurrent nowcasting (LARN) model which trained with a two-step training approach. The first training stage is designed as a lightning augmented pretraining (LAP) module based on self-supervised learning method, which can focus on the critical lightning events to solve the heavy class imbalance problem. The second training stage is designed as a multi-modal data fusion module (MDF), which can effectively fuse lightning, radar and satellite observations to nowcasting lightning. The results of experimental evaluations demonstrate the performance of LARN model outperforms the existing nowcasting models with lead times for up to 90 min. The ablation study shows that the two training stages cooperate well, with the LAP module improving the hit rate and the MDF module reducing the false alarm rate. For the radar and satellite modalities, the vertically integrated liquid (VIL) exhibits the most informative power for lightning nowcasting, followed by 10.7 μm brightness temperatures (IR107) and then 6.9 μm brightness temperatures (IR069). Case studies show that the LARN model can better predict the lightning evolution under different type of thunderstorms . Since the LARN model can reflect the lightning distribution in the reality scenes without adopting under-sampling strategy and subjective loss function design, therefore it can apply to different lightning datasets. • A self-supervised learning method is proposed to solve the heavy class imbalance problem for lightning nowcasting. • A multi-modal information fusion method is proposed to obtain the features more related to lightning nowcasting. • The importance of different modalities for lightning nowcasting is investigated. • The proposed model can nowcast the initiation, growth and decay of lightning activity.