Transition probabilities between synoptic weather types as a fingerprint for climate model evaluation

Global Climate Models (GCMs) generally exhibit significant biases in the representation of large-scale atmospheric circulation. Even after bias adjustment, these errors remain and are inherited to some extent by the derived downscaling products, impairing the credibility of future regional projections. 

We perform a process-based evaluation of state-of-the-art GCMs from CMIP5 and CMIP6, with a focus on the simulation of the synoptic climatological patterns having a most prominent effect on the European climate. To this aim, we use the Lamb Weather Type Classification (LWT, Lamb, 1972). We undertake a comprehensive assessment based on several evaluation measures, such as Kullback-Leibler divergence (KL), Relative Bias and Transition Probability Matrix Score (TPMS), used to assess the ability of the GCMs in reproducing not only the frequencies of the different Lamb Weather Types (LWTs), but also the daily probabilities of transitions among them. We show that the novel TPMS score poses a stringent test on the GCM performance, allowing for a convenient model ranking based on each model’s transition probability matrix fingerprint. Deficiencies in the transition probabilities from one LWT to another might explain the misrepresentation of the synoptic conditions and their frequencies by the GCMs. Four different reanalysis products of varying characteristics are considered as pseudo-observational reference in order to assess observational uncertainty. 

Our results unveil an overall improvement of salient atmospheric circulation features of CMIP6 with respect to CMIP5, demonstrating the ability of the new models to better capture key synoptic conditions. The improvement is consistent across observational references, although it is uneven across models and large frequency biases still remain for the dominant LWTs in many cases. In particular, some CMIP6 models attain similar or even worse results than their CMIP5 counterparts. In light of the large differences found across models, we advocate for a careful selection of driving GCMs in downscaling experiments with a special focus on large-scale atmospheric circulation aspects.