Monitoring and Benchmarking of Earth System Model Simulations with ESMValTool

Earth system models (ESMs) are important tools to improve our understanding of present-day climate and to project climate change under different plausible future scenarios. For this, ESMs are continuously improved and extended resulting in more complex models. Particularly during the model development phase, it is important to continuously monitor how well the historical climate is reproduced and to systematically analyze, evaluate, understand, and document possible shortcomings. For this, putting model biases relative to observations into the context of deviations shown by other state-of-the-art models greatly helps to assess which biases need to be addressed with higher priority. Here, we introduce the new capability of the Earth System Model Evaluation Tool (ESMValTool) to monitor running or benchmark existing simulations with observations in the context of results from the Coupled Model Intercomparison Project (CMIP). ESMValTool is an open-source community-developed diagnostics and performance metrics tool for the evaluation and analysis of ESMs. To benchmark model output, ESMValTool calculates metrics such as root mean square error (RMSE) or the coefficient of determination (R²) relative to reference datasets. This is directly compared to the same metric calculated for an ensemble of models such as CMIP6, which provides a statistical measure for the range of values that can be considered typical for state-of-the-art models. Results are displayed in different types of plots such as map plots (using stippling and hatching) or time series (via uncertainty bands). Automatic downloading of CMIP results from the Earth System Grid Federation (ESGF) makes application of ESMValTool for benchmarking of individual model simulations, for example in preparation of CMIP7, easy and very user friendly.