On associating significance levels with temporal changes in empirical orthogonal function analysis: a case study for ENSO

The availability of a large ensemble enables one to evaluate empirical orthogonal functions (EOFs) with respect to the ensemble without relying on temporal variability at all. Variability across the ensemble at any given time is supposed to represent the most relevant probability distribution for climate-related studies, and this distribution is presumably subject to temporal changes in the presence of time-dependent forcing. Such changes may be observable in spatial patterns of ensemble-based EOFs and associated eigenvalues. Unfortunately, estimates of these changes come with a considerable error due to the finite size of the ensemble, so that associating a significance level with the presence of a change (with respect to a null hypothesis about the absence of any change) should be the first step of analyzing the time evolution.

It turns out, however, that the conditions for the applicability of usual hypothesis tests about stationarity are not satisfied for the above-mentioned quantities. What proves to be feasible is to estimate an upper bound on the significance level for nonstationarity. This means that the true significance level would ideally be lower or equal to what is estimated, which would prevent unjustified confidence in the detection of nonstationarity (i.e., falsely rejecting the null hypothesis could not become more probable than claimed). Most importantly, one would avoid seriously overconfident conclusions about the sign of the change in this way. Notwithstanding, the estimate for the upper bound on the significance level is also affected by the finite number of the ensemble members. It nevertheless becomes more and more precise for increasing ensemble size and may serve as a first guidance for currently available ensemble sizes.

The details of the estimation are presented in the example of the EOF-based analysis of the El Niño–Southern Oscillation (ENSO) as it appears in the historical and RCP8.5 simulations of the Max Planck Institute Grand Ensemble. A comparison between results including and excluding ensemble members initialized with an incomplete spinup in system components with a long time scale is also given.