The challenge of uncertain observations: Probabilistic verification of decadal predictions with high temporal resolution

EGU abstract 2025

NP5.2 EDI: Advances in statistical post-processing, blending, and verification of deterministic and probabilistic forecasts

The challenge of uncertain observations: Probabilistic verification of decadal predictions with high temporal resolution

Verification plays an important role in the evaluation and the development of climate predictions. With new developments in the field and ever larger availability of computational resources, temporal high resolutions become an option. But we often do not make use of the full temporal distribution and much too often we still rely on temporal averages to reduce the dimensionality of the data to make a verification with common metrics manageable. One of the reasons is the challenge how to verify in an understandable manner probabilistic model predictions with probabilistic, uncertain observations.

Tools for probabilistic verification are available, like the Continuous Rank Probability Score (CRPS), but are often defined for perfect observations. Furthermore, many tools are for the wider community hard to comprehend and are as such often not applied. This poses the question on how to verify predictions on the basis of current imperfect usage of metrics within the field and how to communicate prediction skill in general. 

This contribution will address two main approaches and apply it to the comparison between a decadal prediction and the associated projection (historical simulation), with an assimilation simulation as an observational reference. In the first we will ask how to communicate verification results for a wider community. For this we will look at framing the skill as yearly matchups between the two model results. Basing on the Integrated Quadratic Distance each year determines which model result is closer to the observations and the years how often one result was better than the other leads to our verification result. In a second approach it will be discussed to find modifications of some of the most applied metrics in our field, Anomaly Correlation (ACC) and Root-Mean Square (RMS), towards uncertain observations. While these metrics are imperfect, they allow an easy communication for people already applying them. Differences in their interpretation will be discussed, giving us insights about how uncertain observations change our understanding of a good prediction. We address also significance estimation and it will be highlighted why we need to find easy comprehendible approaches to handle uncertain observations in the future.