Estimating the significance of the added skill from initializations: The case of decadal predictions

A considerable part of the skill in decadal forecasts often come from the forcings which are present in both initialized and un-initialized model experiments. This makes the added value from initialization difficult to assess. We investigate statistical tests to quantify if initialized forecasts provide skill over the un-initialized experiments. We consider three correlation based statistics previous used in the literature. The distributions of these statistics under the null-hypothesis that initialization has no added values are calculated by a surrogate data method. We present some simple examples and study the statistical power of the tests. We find that there can be large differences in both the values and the power for the different statistics. In general the simple statistic defined as the difference between the skill of the initialized and uninitialized experiments behaves best. However, for all statistics the risk of rejecting the true null-hypothesis is too high compared to the nominal value.

We compare the three tests on initialized decadal predictions (hindcasts) of near-surface temperature performed with a climate model and find evidence for a significant effect of initializations for small lead-times. In contrast, we find only little evidence for a significant effect of initializations for lead-times larger than 3 years when the experience from the simple experiments is included in the estimation.