Example-Based Concept Analysis Framework for Deep Weather Forecast Models

Abstract To improve the trustworthiness of an artificial intelligence (AI) model, finding consistent, understandable representations of its inference process is essential. This understanding is particularly important in high-stakes operations such as weather forecasting, where the identification of underlying meteorological mechanisms is as critical as the accuracy of the predictions. Despite the growing literature that addresses this issue through explainable AI, the applicability of their solutions is often limited due to their AI-centric development. To fill this gap, we follow a user-centric process to develop an example-based concept analysis framework, which identifies cases that follow a similar inference process as the target instance in a target model and presents them in a user-comprehensible format. Our framework provides the users with visually and conceptually analogous examples, including the probability of concept assignment to resolve ambiguities in weather mechanisms. To bridge the gap between vector representations identified from models and human-understandable explanations, we compile a human-annotated concept dataset and implement a user interface to assist domain experts involved in the framework development. Significance Statement This study investigates deep neural networks’ (DNNs) ability to encode semantic patterns of precipitation mechanisms and aims to provide a ready-to-deploy explainable artificial intelligence (XAI) tool. Key findings reveal that DNNs can extract nonlinear precipitation mechanisms and represent semantically meaningful meteorological attributes. The concept explanations align with expert perceptions, enhancing the interpretability and trustworthiness of model predictions. These findings demonstrate DNNs’ potential to provide insightful, explainable predictions in meteorology, improving the trustworthiness of DNNs for practitioners. Follow-up research could involve refining the XAI framework, exploring its application for other meteorological phenomena, regions or scales, and integrating it with operational systems to assess the strengths and limitations in real-world scenarios.