(Un)fair comparison of classifications of atmospheric circulation patterns between two datasets

Comparison of two different datasets is a common task in climatology. Possible examples include a comparison of two reanalyses; model validation, i.e., a comparison of a climate model output with observed data or a reanalysis; or assessment of future climate change, i.e., a comparison of two model runs for a future and current climate state. Such comparisons may be conducted for a variety of climate elements, including atmospheric circulation. In this contribution, we focus on circulation types as result of a classification of atmospheric circulation patterns.

There are several possible ways of how to conduct the comparison, one of them being a separate and independent classification in either dataset; such a procedure has been used many times.

We take a simple and widely used classification method, k-means clustering, as an example. Two equally long datasets of daily sea level pressure are subjected to k-means clustering, with an equal number of resulting circulation types. Circulation types are then compared between the two datasets. While many types occur in both datasets, there is not a full one-to-one correspondence between the types; i.e., some types occur only in one dataset and some types occur only in the other. The occurrence frequencies of some types substantially differ between the two datasets. Interpretation of some of the identified differences is attempted.

In the next step, we uncover the nature of both datasets: They are subsamples of a single dataset, viz., ERA-5 reanalysis. There is no reason for the two sets of circulation types to differ. All the differences detected (and subsequently interpreted) are artefacts, therefore, not real features. We argue that conducting a classification separately in each dataset is not an appropriate procedure for a fair comparison between two datasets; it is likely to lead to erroneous conclusions. This statement is supported by two other comparison procedures: a single classification applied to both datasets together, and a projection of circulation types identified in one dataset to the other one. They both demonstrate that the circulation types and their frequencies do not in fact differ between the two data subsamples.