Optimizing the computational Performance of the Earth System Model Evaluation Tool with Dask

ESMValTool (Earth System Model eValuation Tool, www.esmvaltool.org) is open-source, community-developed software for the evaluation of Earth system models, mainly in the context of multi-model analyses, e.g. the Coupled Model Intercomparison Project (CMIP). ESMValTool provides a large number of “recipes” that reproduce published figures, for example, some of the figures found in reports of the Intergovernmental Panel on Climate Change (IPCC). ESMValCore, the framework powering ESMValTool, provides capabilities that make it easy to work with data produced for CMIP as well as related observational and reanalysis data, e.g. discovering, downloading, and preprocessing these data.Here, we present how we optimized the computational performance of ESMValTool in the past year, as part of the ESiWACE3 project. To focus our optimization efforts, we defined two distinct use cases (recipes) of the ESMValTool, representing both coarse-scale IPCC-ready CMIP6 multi-model analyses on climatological time scales and shorter seasonal processing jobs for high-resolution climate data. We achieved a speedup factor of 4x for the IPCC-style recipe and a speedup factor of 5x to 10x (depending on the number of compute nodes used) for the high-resolution recipe. We will present how we achieved the speedup and share insights into how to make optimal use of Dask (www.dask.org) for optimizing computational performance. We also give a brief introduction to the CMIP AR7 Fast Track Rapid Evaluation Framework. This framework will be run operationally to publish initial analyses as CMIP7 data becomes available. It builds on existing community evaluation packages, initially ESMValTool, ILAMB/IOMB, and PMP metrics, with the possibility of community expansion after the first release in October 2025. The framework is developed open-source at https://github.com/Climate-REF/.